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(57) The present invention relates to a transcoder 
for executing a re-coding process on an encoded 
stream generated based on an MPEG standard in order 
to generate a re-coded stream having a different GOP 
(Group of Pictures) structure or bit rate. 

Specif icaliy, a decoding device of a transcoder 1 06 
decodes a source encoded stream to generate decoded 
video data and extracts past coding parameters super- 
posed in the encoded stream as history_stream()- In 
this case, the decoding device extracts the past coding 
parameters based on information superposed: ^ the 
encoded stream as re_coding_stream_infqO. "[ : .; 

An encoding device receives the. • d.ecb.d^.' 
data and the past coding parameters ah'd^Wtn^' ^ 



coding parameters to carry out an encoding process in 
a manner such that this process will not degrade image 
quality, thereby generating a re-coded stream. Further, 
the encoding device selects one of the past coding 
parameters which are optimal for an application con- 
nectively following the encoding device and describes 
only the selected past coding parameters in the 
encoded stream as history_streamQ- The encoding 
device superposes, as re_coding_stream_info(), infor- 
mation indicating the selected past coding parameters 
so that the following application can properly extract the 
String parameters for the history_stream() from the re- 
ceded stream. 
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Description 

Technical Field 

[0001] The present invention relates to a coding 
system and method, an encoding device and method, a 
decoding device and method, a recording device and 
method, and a reproducing device and method, and In 
particular, to a coding system and method, an encoding 
device and method, a decoding device and method, a 
recording device and method, and a reproducing device 
and method which are all suitable for use in a trans- 
coder for carrying out a re-coding process on an 
encoded stream that has been encoded in accordance 
with the MPEG standard, to generate a re-coded stream 
having a different GOP (Group of Pictures) structure or 
bit rate. 

Background Art 

[0002] In broadcasting stations for producing and 
broadcasting television programs, the MPEG (Moving 
Picture Experts Group) technique is commonly used to 
compress/encode video data. In particular, in the fields 
of recording of video data on a randomly accessible 
recording medium material such as a tape and trans- 
mission of video data via a cable or a satellite, the 
MPEG technique is becoming a de facto standard. 
[0003] An example of processing carried out In a 
broadcasting station until a video program created 
therein is transmitted to each home will be described in 
brief. First encoder provided in a camcoder comprising 
a video camera and a VTR (Video Tape Recorder) inte- 
grated together is used to encode and record source 
video data on a magnetic tape. In this case, the encoder 
of the camcoder encodes the source video data so as to 
be suited for a recording format for a VTR tape. For 
example, an MPEG bit stream recorded on the mag- 
netic tape has a GOP structure in which one GOP is 
configured by two frames (for example, I, B, I, B, I, B, ...). 
In addition, the MPEG bit streams recorded on the mag- 
netic tape has a bit rate of 1 8 Mbps. 
[0004] Next, a main broadcasting station carries out 
an edition process to edit the video bit stream recorded 
on the magnetic tape. To achieve this, the GOP struc- 
ture of the video bit stream recorded on the magnetic 
tape is converted into one suitable for the edition proc- 
ess. In the GOP structure suited for the edition process, 
one GOP is configured by one frame and all pictures are 
of the I type. This is because the I pictures, which have 
no correlations with other pictures, are most suitable for 
edition in frames. In an actual operation, the video 
stream recorded on the magnetic tape is decoded back 
into baseband video data. This baseband video signal is 
re-coded so that all pictures are converted into the I 
type. By carrying out the decoding and re-coding proc- 
ess in this manner, a bit stream having the GOP struc- 
ture suited for the edition process can be generated. 



[0005] Next, to transmit the edited video program 
generated by the above described edition process, from 
the main station to local stations, the bit stream of the 
edited video program Is converted Into a GOP structure 

5 and bit rate suitable for a transmission process. For 
example, in the GOP structure suited for transmission 
between broadcasting stations, one GOP is configured 
by 15 frames (for example, I, B, B, P, B, B, P, ...). In addi- 
tion, the bit rate suitable for transmissions between 

70 broadcasting stations is preferably high, that is, 50 
Mbps or higher because an exclusive line consisting of 
optical fibers and having a high transmission capacity Is 
commonly provided between broadcasting stations. 
Specifically, the bit stream of the edited video program 

is Is decoded back into the baseband video data. Then, 
the baseband video data is re-coded to have the GOP 
structure and bit rate suited for transmissions between 
broadcasting stations. 

[0006] In a local station, an edition process is car- 
20 ried out to insert commercial films specific to the locality 
into the video program transmitted from the main sta- 
tion. That is, as in the above described edition process, 
the video stream transmitted from the main stream is 
decoded back into the baseband video data. By re-cod- 
25 ing the baseband video signal so that all pictures are 
converted into the I type, a bit stream having the GOP 
structure suited for the edition process can be gener- 
ated. 

[0007] Subsequently, to transmit the video program 
30 edited in the local station to each home via a cable or a 
satellite, conversion into the GOP structure and bit rate 
suited for the transmission process is carried out. For 
example, in the GOP structure suitable for transmis- 
sions to each home, one GOP is configured by 15 
35 frames (for example, I, B, B, P, B, B, P, ...) and the bit 
rate suitable for transmissions to each home is low, that 
is, about 5 Mbps. Specifically, the bit stream of the 
edited video program is decoded back into the base- 
band video data. Then, the baseband video data is re- 
40 coded into the GOP structure and bit rate suitable for 
transmissions. 

[0008] As understood from the above description, 
the plurality of decoding and encoding processes are 
repeated while the video program is being transmitted 

45 from the broadcasting station to each home. In fact, the 
broadcasting station requires various signaling proc- 
esses other than those described above, so that the 
decoding and encoding processes must be repeated for 
each signaling process. 

50 [0009] As is well known, however, the encoding and 
decoding processes based on the MPEG standard are 
not 100% reversible. That is, the baseband video data 
before encoding is not perfectly the same as the 
decoded video data and has its image quality degraded 

55 due to the encoding and decoding processes. Disad- 
vantage ously, repetition of the decoding and encoding 
processes may progressively degrade the image qual- 
ity, as described above. In other words, repetition of the 
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decoding/encoding processes may accumulate degra- 
dation of the image quality. 

[0010] The present invention is provided in view of 
these circumstances, and it is an object of the present 
invention to provide a transcoding system that can pre- 
vent degradation of the image quality even if the decod- 
ing and encoding processes are repeated to change the 
structure of the GOP (Group of Pictures) in the encoded 
bit stream encoded based on the MPEG standard. 

Disclosure of the Invention 

[001 1 ] The present invention relates to a transcoder 
for re-coding an encoded stream generated based on 
the MPEG standard to generate a re-coded stream hav- 
ing a different GOP (Group of Pictures) and a different 
bit rate. 

[0012] Specifically, a decoding device of a trans- 
coder 106 decodes a source encoded stream to gener- 
ate decoded video data and extracts past encoding 
parameters superposed In the encoded stream as 
history_stream(). in this case, the decoding device 
extracts the past encoding parameters based on infor- 
mation superposed in the encoded stream as 
re_coding_stream_info(). 

[0013] An encoding device receives the decoded 
video data and the past encoding parameters and uses 
the latter to encode the decoded video data in such a 
manner as to prevent the re-coding process from 
degrading the image quality, thereby generating a re- 
coded stream. Further, the encoding device selects 
ones of the past encoding parameters which are optimal 
for an application located after the encoding device and 
connected thereto, and writes only the selected past 
encoding parameters into the encoded stream as 
history__stream(). The encoding device superposes 
information indicating the selected past encoding 
parameters, in the encoded stream as 
re_coding_stream_info() so that the following applica- 
tion can adequately extract from the re-coded stream 
the encoding parameters concerning history_stream(). 
[0014] The transcoder according to the present 
invention can realize a coding system that can use a 
minimum number of encoding parameters adapted for 
the following application to minimize degradation of the 
image quality even if video data is repeatedly re-coded. 
[0015] The transcoder according to the present 
invention- comprises -decoding means for decoding a 
source encoded stream to generate video data and 
extracting from the source encoded stream past encod- 
ing parameters generated by a past encoding process, 
encoding means for re-coding the video data to gener- 
ate a re-coded video stream, and control means for 
receiving the past encoding parameters to control the 
re-coding process by the encoding means based on the 
past encoding parameters and selectively writing the 
past encoding parameters into a re-coded stream. 
[0016] The encoding device of the transcoder 



according to the present invention selects ones of the 
past encoding parameters which are required by the 
application located after the encoo5ng means and con- 
nected thereto and writes the selected past encoding 

5 parameters into the re-coded stream. 

[0017] The encoding device of the transcoder 
according to the present invention selectively writes the 
past encoding parameters into the re-coded stream and 
writes into the re-coded stream a flag or/and indicator 

io indicating a data set of the past encoding parameters 
written into the re-coded stream. 
[0018] The encoding device of the transcoder 
according to the present invention writes information on 
the past encoding parameters into the encoded stream 

15 as history_streamO and writes information on the re- 
coded stream Into the re-coded stream as 
re_codingLStream_info(). 

[0019] The encoding device of the transcoder 
according to the present invention selectively writes the 
20 past encoding parameters into the re-coded stream as 
history_stream() and writes into the re-coded stream as 
re_codingLstream_info(), information on a data set of 
the past encoding parameters written into the re-coded 
stream. 

25 [0020] The encoding device according to the 
present invention receives past encoding parameters 
on a past encoding process executed on video data, 
selectively writes the past encoding parameters into a 
re-coded stream, and writes into the re-coded stream, 

30 information indicating a data set of the past encoding 
parameters written into the re-coded stream. 
[0021] The decoding device according to the 
present invention extracts from the encoded stream the 
information on the data set of the past encoding param- 

35 eters superposed in the encoded stream and extracts 
the past encoding parameters from the encoded stream 
based on the information on the data set. 
[0022] The decoding device according to the 
present invention extracts from the encoded stream the 

40 flag or/and Indicator written into the encoded stream as 
re_coding_stream Jnfo() and extracts the past encoding 
parameters from the encoded stream based on the flag 
or/and indicator. 

45 Brief Description of Drawings 

[0023] 

Figure 1 is a diagram for describing the principle of 

50 a high efficient encoding. 

Figure 2 is a diagram for describing how picture 
types are processed in compressing image data. 
Figure 3 is a diagram for describing how the picture 
types are processed in compressing image data. 

55 Figure 4 is a diagram for describing the principle of 
encoding of moving image signals. 
Figure 5 is a block diagram showing the configura- 
tions of devices for encoding and decoding moving 
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image signals. 

Figure 6 is a diagram for describing the configura- 
tion of image data. 

Figure 7 is a block diagram showing the configura- 
tion of an encoder 1 8 in Figure 5. 5 
Figure 8 is a diagram for describing the operation of 
a prediction mode-switching circuit 52 in Figure 7. 
Figure 9 Is a diagram for describing the operation of 
the prediction mode-switching circuit 52 in Figure 7. 
Figure 10 is a diagram for describing the operation io 
of the prediction mode-switching circuit 52 in Figure 
7. 

Figure 1 1 is a diagram for describing the operation 
of the prediction mode-switching circuit 52 in Figure 
7. 15 
Figure 12 is a block diagram showing the configura- 
tion of a decoder 31 in Figure 5. 
Figure 13 is a diagram for describing SNR control 
corresponding to the picture type. 
Figure 1 4 is a block diagram showing the configura- 20 
tion of a transcoder 101 to which the present inven- 
tion is applied. 

Figure 15 is a block diagram showing the configura- 
tion of the transcoder 1 01 in Figure 14 in detail. 
Figure 1 6 is a block diagram showing the configura- 25 
tion of a decoding device 1 02 provided in a decod- 
ing apparatus 102 in Figure 14. 
Figure 17 is a diagram for describing pixels in a 
macroblock. 

Figure 18 is a diagram for describing an area in 30 
which encoding parameters are recorded. 
Figure 1 9 is a block diagram showing the configura- 
tion of an encoding device 106 provided in an 
encoding device 106 in Figure 14. 
Figure 20 is a block diagram showing an example of 35 
the configuration of history VLC21 1 in Figure 15. 
Figure 21 is a block diagram showing an example of 
the configuration of history VLD203 in Figure 15. 
Figure 22 is a block diagram showing an example of 
the configuration of a converter 212 in Figure 15. ao 
Figure 23 is a block diagram showing an example of 
the configuration of a staff circuit 323 in Figure 22. 
Figure 24 is a timing chart for describing the opera- 
tion of the converter 212 in Figure 22. 
Figure 25 is a block diagram showing an example of 45 
the configuration of a converter 202 in Figure 15. 
-Figure 26 is a block diagram showing an example of 
the configuration of a delete circuit 343 in Figure 25. 
Figure 27 is a block diagram showing another 
example of the configuration of the converter 21 2 in so 
Figure 1 5. 

Figure 28 is a block diagram showing another 
example of the configuration of the converter 202 in 
Figure 1 5; 

Figure 29 is a block diagram showing an example of 55 
the configuration of a user data formatter 213 in 
Figure 1 5. 

Figure 30 is a diagram showing how the transcoder 



A1 6 

101 in Figure 14 is actually used. 

Figure 31 is a diagram for describing an area in 

which encoding parameters are recorded. 

Figure 32 is a flowchart for describing a changeable 

picture type determining process carried out by the 

encoding device 106 in Figure 14. 

Rgure 33 is a diagram showing an example of a 

change In picture type. 

Rgure 34 is a diagram showing another example of 
a change in picture type. 

Rgure 35 is a diagram for describing a quantization 
controlling process carried out by the encoding 
device 106 in Rgure 14. 

Rgure 36 is a flowchart for describing the quantiza- 
tion controlling process carried out by the encoding 
device 1 06 in Rgure 1 4. 

Rgure 37 is a block diagram showing the configura- 
tion of a tightly coupled transcoder 1 01 . 
Rgure 38 is a diagram for describing a syntax for a 
stream of a video sequence. 
Rgure 39 is a diagram for describing the configura- 
tion of the syntax in Rgure 38. 
Rgure 40 is a diagram for describing a syntax for 
history_streamO having history information of a 
fixed length recorded therein. 
Rgure 41 is a diagram for describing the syntax for 
history_stream() having the history information of 
the fixed length recorded therein: 
Rgure 42 is a diagram for describing the syntax for 
history_streamO having the history information of 
the fixed length recorded therein. 
Rgure 43 is a diagram for describing the syntax for 
history_stream() having the history information of 
the fixed length recorded therein. 
Figure 44 is a diagram for describing the syntax for 
history_stream() having the history information of 
the fixed length recorded therein. 
Rgure 45 is a diagram for describing the syntax for 
history_streamO having the history information of 
the fixed length recorded therein. 
Figure 46 is a diagram for describing the syntax for 
history_stream() having the history information of 
the fixed length recorded therein. 
Rgure 47 is a diagram for describing the syntax for 
history_streamO having the history information of 
the variable length recorded therein. 
Rgure 48 is a diagram for describing a syntax for 
sequence_headerO- 

Rgure 49 is a diagram for describing a syntax for 
se q u ence_exte nsion (). 

Rgure 50 is a diagram for describing a syntax for 

exte nsion_and_user_data(). 

Rgure 51 is a diagram for describing a syntax for 

user_dataQ. 

Figure 52 is a diagram for describing a syntax for 

group_of_pictures_header(). 

Rgure 53 is a diagram for describing a syntax for 

picture_header(). 
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Figure 54 is a diagram for describing a syntax for 
picture^coding^extension (). 

Figure 55 is a diagram for describing a syntax for 
extension_data(). 

Figure '56 Is a diagram for describing a syntax for s 
quant_matrix_extension()- 

Figure 57 is a diagram for describing a syntax for 
copyright_extension(). 

Figure 58 is a diagram for describing a syntax for 
plcture_dlspIay_extenslon(). to 
Figure 59 is a diagram for describing a syntax for 
picture_data(). 

Figure 60 is a diagram for describing a syntax for 
sfice(). 

Figure 61 is a diagram for describing a syntax for 15 
macroblock(). 

Figure 62 is a diagram for describing a syntax for 
macroblock^modesO. 

Figure 63 is a diagram for describing a syntax for 
motion_vectors(s). 20 
Figure 64 is a diagram for describing a syntax for 
motion_vector(r, s). 

Figure 65 is a diagram for describing variable- 
length codes of macroblock_type for I pictures. 
Figure 66 is a diagram for describing variable- 25 
length codes of macroblock_type for P pictures. 
Figure 67 is a diagram for describing variable- 
length codes of macroblock_Jype for B pictures. 
Figure 68 is a diagram for describing a syntax for 
re_coding_stream_info(). 30 
Figure 69 is a diagram for describing red_bw_fiag 
and red_bw_indicator. 

Figure 70 is a diagram for describing a data set of 

history information encoding parameters. 

Figure 71 is a diagram for describing Re_coding 35 

Information Bus macroblock formation. 

Figure 72 is a diagram for describing Picture rate 

elements. 

Figure 73 is a diagram for describing Picture rate 
elements. 40 
Figure 74 is a diagram for describing Picture rate 
elements. 

Figure 75 is a diagram for describing an area in 
which Re_Coding Information Bus is recorded. 
Figure 76 is a block diagram representing an exam- 45 
pie of the configuration of a video tape recorder 
- — recording system. 

Figure 77 is a block diagram representing an exam- 
ple of the configuration of a video tape recorder 
reproducing system. so 
Figure 78 is a block diagram representing another 
example of the configuration of the video tape 
recorder recording system. 

Figure 79 is a block diagram representing an exam- 
ple of the configuration of the video tape recorder 55 
reproducing system. 

Figure 80 is a diagram for describing the recording 
positions of a video stream and history_stream. 



Best Mode for Carrying Out the Invention 

[0024] Before a transcoder to which the present 
invention is applied is described below, compressive 
encoding of moving Image signals will be explained. 
The terms used for a system herein refer to the entire 
system configured by plural devices and means. 
[0025] For example, television conference or tele- 
phone systems that transmit moving image signals to 
remote sites use line or interframe correlatJonshtp 
among video signals to compressive-code image sig- 
nals in order to efficiently use transmission paths. 
[0026] The use of the line correlationship enables 
image signals to be compressed, for example, by 
means of a DCT (Discrete Cosine Transformation) proc- 
ess. 

[0027] In addition, the use of the interframe correla- 
tionship enables the image signals to be further com- 
pressed for encoding. As shown, for example, in Figure 
1, if frame images PC1 to PC3 occur during points of 
time t1 to t3, the difference In image signal between the 
frame images PC1 and PC2 is calculated to generate 
PC12, and the difference between frame images PC2 
and PC3 is calculated to generate PC23. Images in 
temporally adjacent frames do not typically have sub- 
stantially large variations, so that calculation of the dif- 
ference therebetween results in a differential signal of a 
small value. Accordingly, this differential signal can be 
used for encoding to compress the amount of encoding. 
[0028] However, transmission of only differential 
signals does not allow the restoration of the original 
images. Thus, the image in each frame is encoded into 
one of three picture types including I, P, and B pictures 
in order to compressive-code the image signals. 
[0029] That is, as shown, for example, in Figure 2, 
the image signals for 17 frames including frames F1 to 
F17 are grouped into a group of pictures (GOP), which 
is a processing unit. Then, the image signal in the frame 
F1 is encoded into the I picture, the second frame F2 is 
encoded Into the B picture, and the third frame F3 is 
encoded into the P picture. The fourth and subsequent 
frames F4 to F17 are alternately encoded into the B or 
P picture. 

[0030] For the I picture image signal, the image sig- 
nal for the corresponding frame is transmitted as it is. 
On the contrary, for the P picture image signal, basically, 
the differential between this image signal and the tem- 
porally preceding I or P picture is transmitted as shown 
in Figure 2. Further, for the B picture image signal, basi- 
cally, the differential between this image signal and the 
average of the temporally preceding and following 
frames is encoded, as shown in Rgure 3. 
[0031] Figure 4 shows the principle of a method of 
encoding moving image signals. As shown in this figure, 
since the first frame F1 is processed as the I picture, it 
is transmitted to a transmission path as transmitted data 
F1 X (intraimage encoding). On the contrary, since the 
second frame F2 is processed as the B picture, the dif- 
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ferential between this frame and the average of the tem- 
porally preceding frame. F1 and the temporally following 
frame F3 is calculated and transmitted as transmitted 
data F2X. , . 

[0032] In fact, there are four types of processing for s 
the B pictures; A first type of processing transmits^ the 
data in the original frame "F2 as the transmitted data 
F2X without any calculation (SP1) (iritracoding); this 
process! ng is similar to that for-the I pictures. A second 
type of process! hg calculates and trahsm its the differen- io 
tial (SP2) between the original frame F2 and trtetfempo-: 
rally following frame ^ F3 (baclward prealctrve encoding)^ 
A third type of processing transmits; the differential 
(SP3) between theoriginal frame F2?and the temporally 
preceding frame F1 (forward predictive encoding): Fur- is 
ther, a fourth type; of processing generates the differen- 
tial (SP4> between the original frame F2 and the 
average of the temporally preceding frame F1 and tem- 
porally following frame F3 and transmits it as the trans- 
mitted data F2X (bidirectional predictive encoding). 20 
[0033] One of the four methods described above 
which transmits the smallest amount of data is actually 
employed. 

[0034] In transmitting the differential data, a motion 
vector x1 (between the frames F1 and F2) (in the case 25 
of forward prediction) or x2 (between the frames F3 and 
F2) (in the case of backward prediction) or both (in the 
case of bidirectional prediction) between the original 
frame and the image (reference image) in the frame for 
which the differential with the original frame is to be cal- 30 
culated is also transmitted. 

[0035] In addition, for the P picture frame F3, the 
temporally preceding frame F1 is used as the reference 
image to calculate the differential signal (SP3) between 
the frames F3 and F1 as well as the motion vector x3, 35 
both of which are transmitted as transmitted data F3X 
(forward predictive encoding). Alternatively, the data in 
the original frame F3 is transmitted as the data F3X 
(SP1) (intracoding). As in the B pictures, one of these 
methods which transmits the smaller amount of data is 40 
selected. 

[0036] Figure 5 shows an example of the configura- 
tion of devices for encoding and transmitting moving 
image signals and decoding them based on the above 
described principle. An encoding device 1 is adapted to 45 
encode and transmit input video signals to a recording 
medium 3~ac1ing as a transmission path. A decoding " 
devicer2:lsvadapted torreproduce-therslgnals recorded - 
on the recording medium 3 and decode and output 
them. so 
[0037] In an encoding device 1; Input video signals 
are input to a preprocess circuit 11,. where the images 
are each separated into a brightness signal and a color 
signal (in this embodiment, a color difference signal). 
The brightness and color difference signals, which are 55 
analog, are converted into digital signals by A/D con- 
verters 12, 13, respectively. The digital signals into 
which the video signal has been converted by the A/D 



converters 12, 13 are supplied to a frame memory 14 for 
storage. The frame memory 14 stores the brightness 
signal in a brightness signal frame memory 15, while 
storing the color difference signal in a color difference 
signal frame memory 16. 

[0038] A format converting circuit 1 T converts the ; 
signals stored in the frame riiemory 14 in a frame for- 
mat;' into a block format That ls, ; as shown in Figure 6, 
the video signals stored in the frame memory 14 are of 
the frame' format shown In Figure 6(A), which is config- 
ured by \Mines each composed of H dots. The format 
converting circuit 17 partitions the signal for one frame 
Into M slices each configured by 16 lines as shown in 
Figure 6(B). Each slice- is divided into M macroblocks. 
As shown in Figu re 6(G), the macroblock is configured 
by a brightness signal corresponding to 16 x 16 pixels 
(dots), and this brightness signal is further partitioned 
into blocks Y[1 ] to Y[4] each configured by 8 x 8 dots. 
The brightness signal of the 16 x 16 dots corresponds 
to a Gb signal of 8x8 dots and a Cr signal of 8 x 8 dots. 
[0039] In this manner, the data converted into the 
block format is supplied by the format converting circuit 
1 7 to an encoder 18, where encoding is carried out. The 
details will be described with reference to Figure 7. 
[0040] A signal obtained through encoding by the 
encoder 18 is output to a transmission path as a bit 
stream. For example, the signal is supplied to a record- 
ing circuit 19 and recorded on the recording medium 3 
as a digital signal. 

[0041] Data reproduced from the recording medium 
3 by a reproduction circuit 30 of the decoding device 2 is 
supplied to a decoder 31 for decoding. The details of the 
decoder 31 will be described below with reference to 
Figure 12. 

[0042] Data obtained through decoding by the 
decoder 31 is input to a format converting circuit 32, 
where the block format is converted into the frame for- 
mat Then, the brightness signal in the frame format is 
supplied to a brightness signal frame memory 34 of a 
frame memory 33 for storage, and the color difference 
signal is supplied to a color difference frame memory 35 
for storage. The brightness and color difference signals 
are read out from the brightness signal frame memory 
34 and the color difference signal frame memory 35, 
respectively, and subsequently converted - by D/A con- 
verters 36; 37 into analog signals, which, are then sup- 
plied tore postprocess-circuit 38: The rpostprocess circuit 
38 synthesizes-the-brightnessTandTColor difference sigr 
nals before output 

[0043] Next, the configuration of the encoder 1 8 will 
be described with reference to Figure 7. Image data to 
be encoded is input to a motion vector-detecting circuit 
50 in terms of macroblocks. The motion vector-detect- 
ing circuit 50 processes the image data in each frame 
as the I, P, or B picture in accordance with a preset pre- 
determined sequence. Whether the sequentially input 
image in each frame is processed as the I, P, or B pic- 
ture is determined beforehand (for example, the group 
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the quantization scale supplied by the variable-length 
decoding circuit 82, to output the inversely quantized 
data to an IDCT circuit 84. The data (DCT coefficient) 
output by the inverse-quantization circuit 83 is subjected 
to inverse discrete cosine transformation by the IDCT 
circuit 84 and the processed data is delivered to a calcu- 
lator 85. 

[0093] If the image data delivered to the calculator 

85 by the IDCT circuit 84 is for the I picture, it is output 
by the calculator 85 and supplied to a forward-predicted 
image section 86a of a frame memory 86 for storage in 
order to generate predicted image data for the image 
data (the P or B picture data) Input to the calculator 85 
later. Additionally, this data is output to the format con- 
verting circuit 32 (Figure 5). 

[0094] If the image data delivered by the IDCT cir- 
cuit 84 is for the P picture, which uses the image data in 
the preceding frame as the predicted image data, arid is 
in the forward prediction mode, then the image data (the 
I picture data) in the preceding frame stored in the for- 
ward-predicted image section 86a of the frame memory 

86 is read out and subjected to motion compensation by 
the motion compensating circuit 87 in a fashion corre- 
sponding to the motion vector output by the variable- 
length decoding circuit 82. The resulting data is added 
by the calculator 85 to the image data (differential data) 
supplied by the IDCT circuit 84, and the data thus 
obtained is output. The data resulting from the addition, 
that Is, the decoded P picture data Is supplied to a back- 
ward-predicted image section 86b of the frame memory 
86 for storage in order to generate predicted image data 
for the image data (the B or P picture data) Input to the 
calculator 85 later. 

[0095] Like the I picture data, data in the intraimage 
prediction mode is stored in the backward-predicted 
image section 86b without being processed by the cal- 
culator 85, even if it is for the P picture. 
[0096] This P picture is to be displayed after the 
next B picture, so that it is not output to the format con- 
verting circuit 32 at this point (as described above, the P 
picture input after the B picture is processed and trans- 
mitted before the B picture). 

[0097] If the image data delivered by the IDCT cir- 
cuit 84 is for the B picture, the i picture image data 
stored in the forward-predicted image section 86a of the 
frame memory 86 (in the case of the forward prediction 
mode), the P picture image data stored in the backward- 
predicted image section 86b of the frame memory 86 (I n- 
the case of the backward prediction mode), or both of 
these image data (in the case of the bidirectional predic- 
tion mode) is read out depending on the prediction 
mode supplied by the variable-length decoding circuit 
82, and is then subjected to motion compensation by 
the motion compensating circuit 87 in a fashion corre- 
sponding to the motion vector supplied by the variable- 
length decoding circuit 82. A predicted image is thereby 
generated. No predicted image, however, is generated if 
no motion compensation is required (in the case of the 



intraimage prediction mode). 

[0098] The data subjected to the motion compensa- 
tion by the motion compensating circuit 87 in the above 
manner is added by the calculator 85 to the output from 
5 the IDCT circuit 84. This addition output is provided for 
the format converting circuit 32. 

[0099] This addition output, however, comprises the 
B picture data and is not used to generate a predicted 
image for other images. Thus, it is not stored in the 

10 frame memory 86. . 

[0100] After the outputting of the B picture image, 
the P picture image data stored in the backward-pre- 
dicted Image section 86b Is read out and supplied to the 
calculator 85 via the motion compensating circuit 87. 

15 However, no motion compensation is provided at this 
point 

[0101] Although the illustrated decoder 31 has no 
circuit corresponding to the prediction mode-switching 
circuit 52 and DCT mode-switching circuit 55 in the 
20 encoder 1 8 in Figure 5, the motion compensating circuit 
87 carries out processing corresponding to these cir- 
cuits, that is, one for recovering the configuration in 
which odd-number field line signals are separated from 
even-number field line signals, to the original configure- 
rs tion. 

[01 02] In addition, although the above description is 
for the processing of the brightness signals, the color 
difference signals are similarly processed. In this case, 
however, the motion vector is obtained by reducing the 

30 motion vector for the brightness signals to half in both 
vertical and horizontal directions. 
[0103] Figure 13 shows the quality of encoded 
images. The quality (SNR: Signal to Noise Ratio) of 
images is controlled correspondently to the picture type 

35 in a manner such that I and P pictures are both of a high 
quality, while the B picture has a lower quality than the I 
and P pictures. This method utilizes human visual char- 
acteristics; that is, a higher visual quality is obtained by 
vibrating the quality of each image than by leveling out 

40 the quality of all images. The control of image quality 
depending on the picture type is executed by the quan- 
tization circuit 57 in Figure 7. 

[0104] Figures 14 and 15 show the configuration of 
a transcoder 101 to which the present invention is 

45 applied; Figure 15 shows the detailed configuration of 
Figure 1 4. The transcoder 1 01 Converts the GOP struc- 
ture and bit rate of an encoded video bit stream input to 
a decoding device 1 02 into those desired by an opera- 
tor. To explain the functions of the transcoder 101, three 

so transcoders having functions similar to those of the 
transcoder 101 are assumed to be connectively located 
before the transcoder 101, though not shown in Figure 
1 5. That is, in order to vary the GOP structure and bit 
rate of a bit stream, the first, second, and third transcod- 

55 ers are connected in series in this order, and a fourth 
transcoder, shown in Figure 15, is connectively located 
after the third transcoder. 

[0105] In the following description of the present 
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invention, an encoding process carried out in the first 
transcoder is defined as a first-generation encoding 
process, one in the second transcoder, which is connec- 
tively located after the first transcoder, is defined as a 
second-generation encoding process, one in the third 
transcoder, which is connectively located after the sec- 
ond transcoder, Is defined as a third-generation encod- 
ing process, and one in the fourth transcoder (the 
transcoder 101 shown in Figure 15), which is connec- 
tively located after the third transcoder, is defined as a 
fourth-generation encoding process or current encoding 
process. 

[0106] In addition, encoding parameters generated 
during the first-generation encoding process are called 
"first-generation encoding parameters," encoding 
parameters generated during the second-generation 
encoding process are called "second-generation 
encoding parameters," encoding parameters generated 
during the third-generation encoding process are called 
"third-generation encoding parameters," and encoding 
parameters generated during the fourth-generation 
encoding process are called "fourth-generation encod- 
ing parameters." 

[0107] First, an encoded video stream ST(3rd) sup- 
plied to the transcoder 101 shown in Figure 15 will be 
described. The ST(3rd) represents a third-generation 
encoded stream generated during the third-generation 
encoding process carried out by the third transcoder 
provided before the transcoder 101. In the encoded 
video stream ST(3rd) generated during the third-gener- 
ation encoding process, the third-generation encoding 
parameters generated during the third encoding proc- 
ess are described in a sequence layer, a GOP layer, a 
picture layer, a slice layer, and a macroblock layer of this 
encoded video stream ST (3rd) as a 
sequence_header() function, a sequence_extension() 
function, a group_of__pictures_header() function, a 
picture_header() function, a picture_coding_extension() 
function, a picture_data{) function, a slice () function, 
and a macroblockO function. The description of the third 
encoding parameters used during the third encoding 
process in the third encoded stream generated during 
the third encoding process is defined in the MPEG2 
standard and has no novelty. 

[0108] The transcoder 1 01 according to the present 
invention is unique in that the third encoded stream 
ST(3rd) has described therein the third encoding 
parameters as well as the first- and second- gene ration 
encoding parameters generated during the first and 
second encoding processes, respectively. 
[0109] Specifically, the first- and second- gene ration 
encoding parameters are described in a user data area 
of the picture layer of the third-generation encoded 
video stream ST(3rd) as a history stream 
history__stream(). In the present invention, the history 
stream described in the user data area of the picture 
layer of the third-generation encoded video stream 
ST(3rd) is called "history information," and the encoding 



parameters described in the history stream is called 
"history parameters." 

[01 1 0] Alternatively, if the third-generation encoding 
parameters described in the third-generation encoded 

5 stream ST(3rd) are called the "current encoding param- 
eters," since the first- and second-generation encoding 
processes are carried out before the third-generation 
process, the encoding parameters described as history 
stream in the user data area of the picture layer of the 

io third-generation encoded stream ST(3rd) are also 
called "past encoding parameters." 
[0111] The reason why the third encoded stream 
ST(3rd) has described therein the third encoding 
parameters as well as the first- and second-generation 

is encoding parameters generated during the first and 
second encoding processes, respectively, as described 
above is that the image quality can be prevented from 
degradation even with repeated changes in the GOP 
structure or bit rate of the encoded stream through a 

20 transcoding process. 

[01 1 2] For example, it Is contemplated that a picture 
may be encoded into the P type during the first-genera- 
tion encoding process, that this P picture may be 
encoded Into the B type during the second-generation 

25 encoding process in order to change the GOP structure 
of the first-generation encoded stream, and that this B 
picture may be encoded back into the P type during the 
third-generation encoding process in order to further 
change the GOP structure of the second-generation 

30 encoded stream. It is known that since the encoding 
and decoding processes based on the MPEG standard 
are not 100% reversible, the Image quality may be 
degraded with repetition of these processes. 
[0113] In this case, the encoding parameters such 

35 as the quantization scale, the motion vector, and the 
prediction mode which have been generated during the 
first-generation encoding process are reused during the 
third-generation encoding process instead of calculat- 
ing these encoding parameters again during the latter 

40 process. The encoding parameters such as the quanti- 
zation scale, the motion vector, and the prediction mode 
which have been generated by the first-generation 
encoding process are evidently more accurate than 
these parameters newly generated by the third-genera- 

45 tion encoding process, so that the reuse of the first-gen- 
eration parameters can lessen degradation of the image 
quality despite repetition of the encoding and decoding 
processes. 

[0114] The above described processing according 
50 to the present invention will be described in further 
detail by taking, by way of example, processing carried 
out by the fourth-generation transcoder 1 01 shown in 
Figure 15. 

[0115] A decoding device 1 02 uses the third-gener- 
55 ation encoding parameters to decode encoded videos 
from the third-generation encoded bit stream ST(3rd) 
and generates digital video data for the decoded base- 
bands. Further, the decoding device 102 also decodes 
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the first and second encoding parameters described as 
history stream in the user data area of the picture layer 
of the third-generation encoded bit stream ST(3rd). 
[0116] Specifically, as shown in Figure 16, the 
decoding device 1 02 is configured basically in the same 
manner as the decoder 31 (Figure 12) of the decoding 
device 2 in Figure 5; it comprises a reception buffer 81 
for buffering a supplied bit stream, a variable-length 
decoding circuit 112 for subjecting the encoded bit 
stream to variable-length decoding, an inverse-quanti- 
zation circuit 83 for inversely quantizing, the variable- 
length-decoded data in accordance with the quantiza- 
tion scale supplied from the variable-length decoding 
circuit 112, an IDCT circuit 84 for subjecting the 
inversely quantized DCT coefficient to inverse discrete 
cosine transformation, and a calculator 85, a frame 
memory 86, and a motion compensating circuit 87 for 
carrying out a motion compensating process. 
[0117] To decode the third-generation encoded bit 
stream ST (3rd), the variable-length decoding circuit 
112 extracts the third-generation encoding parameters 
described in the picture, slice, and macroblock layers of 
the third-generation encoded bit stream ST(3rd). For 
example, the third-generation encoding parameters 
extracted by the variable-length decoding circuit 112 
include picture_coding_type indicating the picture type, 
quantiser_scale_code indicating a quantization scale 
step size, macroblockjype indicating the prediction 
mode, motion_vector indicating the motion vector, 
frame/fie ld_motion_type indicating the frame prediction 
mode or the field prediction mode, dctjype indicating 
the frame DCT mode or the field DCT mode, etc. The 
quantiser_scale_code extracted by the variable-length 
decoding circuit 1 12 is supplied to the inverse-quantiza- 
tion decoding circuit 83, and the parameters such as the 
picture_coding_type, quantiser_scale_code, 
macroblockjype, motion_vector, 
frame/Tie I d_motion_type, and dct_type are delivered to 
the motion compensating circuit 87. 
[0118] The variable-length decoding circuit 112 
extracts from the sequence, GOP, picture, slice, and 
macroblock layers of the third-generation encoded bit 
stream ST(3rd) not only the above encoding parameters 
required to decode the third-generation encoded bit 
stream ST(3rd) but also encoding parameters to be 
transmitted to the following fifth-generation transcoder 
as third-generation history information. Of course, the 
third-generation encoding parameters such as the 
picture_codingJype, quantiser_scale_code, 
macroblock_type, motion_vector. 
frame/Reld_motion_type, and dct_type which are used 
for the third-generation decoding process are included 
in the third-generation history information. Encoding 
parameters to be extracted as the history information 
are preset by the operator or a host computer depend- 
ing on the transmission capacity. 
[0119] Furthermore, the variable-length decoding 
circuit 1 1 2 extracts user data described in the user data 



area of the picture layer of the third-generation encoded 
bit stream ST(3rd), to supply this data to a history 
decoding device 1 04. 

[0120] The history decoding device 104 is a circuit 

5 for extracting the first- and second-generation encoding 
parameters described as history information (encoding 
parameters preceding the preceding-generation encod- 
ing parameters) from the user data described In the pic- 
ture layer of the third-generation encoded bit stream 

to ST(3rd). Specifically, the history decoding device 104 
can analyze the syntax of the received user data to 
detect a unique Histrby_Data_ld described in the user 
data, thereby extracting converted JiIstory_stream(). 
Further, the history decoding device 1 04 can obtain 

is history_streamO by removing 1-bit marker bits 
(marker__bit) Inserted Into the 

converted JVistory_stream() at predetermined intervals 
and can analyze the syntax of the history_stream() to 
obtain the first- and second-generation encoding 

20 parameters described in the history_stream(). The 
detailed description of the history decoding device 104 
will be described later. 

[0121] A history information-multiplexing device 
103 is a circuit for multiplexing the first-, second-, and 

25 third-generation encoding parameters on baseband 
video data decoded by the decoding device 102 in order 
to supply these parameters to an encoding device 106 
that carries out the fourth-generation encoding process. 
Specifically, the history information-multiplexing device 

30 103 receives the baseband video data output from the 
calculator 85 of the decoding device 102, the third-gen- 
eration encoding parameters output from the variable- 
length decoding device 1 1 2 of the decoding device 1 02, 
and the first-and second-generation encoding parame- 

35 tens output from the history decoding device 104, to 
multiplex these first-, second-, and third-generation 
encoding parameters on the baseband video data. The 
baseband video data multiplexed with the first-, second- 
, and third-generation encoding parameters is supplied 

40 to a history information-separating device 105 via a 
transmission cable. 

[0122] Next, the method for multiplexing the first-, 
second-, and third-generation encoding parameters on 
the baseband video data will be described with refer- 

45 ence to Figures 17 and 1 8. Figure 17 shows a macrob- 
lock consisting of 1 6 x 1 6 pixels as defined in the MPEG 
standard. The macroblock of 16 x 16 pixels is config- 
ured by four subblocks (Y[0], [1], [2], and Y[3J) for the 
brightness signals each consisting of 8 x 8 pixels and 

so four subblocks (Cr[0], r[1], b[0], and Cb[1]) for the color 
difference signals each consisting of 8 x 8 pixels. 
[0123] Figure 18 shows a certain format of video 
data. This format is defined in ITU Recommendation- 
RDT601 and represents what is called the "D1 format,* 

55 which is used in the broadcasting industry. The D1 for- 
mat is standardized for transmission of 1 0-bit video data 
and thus enables 1 pixel of video data to be expressed 
with 10 bits. 
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[0124) Since baseband video data decoded in 
accordance with the MPEG standard is configured by 8 
bits, the transcoder according to the present invention 
uses the upper eight of the 10 bits in the D1 format (D9 
to D2) as shown in Figure 18, to transmit the baseband 
video data decoded based on the MPEG standard. 
When the 8-bit video data thus decoded Is written into 
the D1 format, the lower two bits (D1 and DO) are unal- 
located. The transcoder according to the present Inven- 
tion uses this unallocated area to transmit the history 
information. 

[0125] Since the data block described in Figure 18 
- Is used to transmit one pixel in each subblock (Y[0], 
Y[1]. V[2], Y[3], Cr[0], Cr[1J, Cb[0], Cb[1]), 64 data 
blocks such as that shown in Figure 18 are transmitted 
in order to transmit one macroblock of data. The use of 
lower two bits (D1 and DO) enables 1 024 (= 1 6 x 64) bits 
of history information to be transmitted for one macrob- 
lock of video data. Accordingly, since history information 
for one generation is configured by 256 bits, history 
information for past four (= 1024/256) generations can 
be superposed on one macroblock of video data. In the 
example shown in Figure 18, the first-, second-, and 
third-generation history information is superposed ther- 
eon. 

[0126] The history information-separating device 
105 is a circuit for extracting the baseband video data 
from the upper eight bits of the data transmitted in the 
D1 format while extracting the history information from 
the lower two bits. In the example shown in Figure 15, 
the history information-separating device 105 extracts 
the baseband video data from the transmitted data, sup- 
plies this data to an encoding device 106, executes the 
first-, second-, and third-generation history information 
from the transmitted data, and supplies the information 
to the encoding device 106 and a history encoding 
device 1 07. 

[0127] The encoding device 106 encodes the base- 
band video data supplied from the history information- 
separating device 105 into a bit stream having a GOP 
structure and bit rate specified by the operator or the 
host computer. Changing the GOP structure means 
changing the number of pictures contained in the GOP, 
the number of P pictures present between I pictures, 
and the number of B pictures present between I pictures 
and P pictures (or I pictures). 

[0128] In the example shown in Figure 15, since the 
supplied baseband video data has the first-, second-, 
and third-generation history information superposed 
thereon, the encoding device 106 selectively reuses the 
history information for an encoding process for the 
fourth generation in order to restrain possible image 
degradation stemming from the re-coding process. 
[0129] Figure 19 is a diagram showing a specific 
configuration of an encoding device 106 provided in the 
encoding device 106. The encoding device 106 Is basi- 
cally configured similarly to the encoder 18 shown in 
Figure 7; it comprises the motion vector-detecting circuit 
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50, the frame/Field prediction mode-switching circuit 52, 
the calculator 53, the DCT mode-switching circuit 55, 
the DCT circuit 56, the quantization circuit 57, the varia- 
ble-length encoding circuit 58, the transmit buffer 59. 

5 the inverse-quantization circuit 60, the inverse-DCT cir- 
cuit 61 , the calculator 62, the frame memory 63, and the 
motion compensating circuit 64. Since the functions of 
each of the circuits are almost the same as in the 
encoder 18 described in Figure 7. description thereof is 

io omitted. The following description will focus on differ- 
ences between the encoding device 106 and the 
encoder 18 described In Figure 7. 
[0130] The encoding device 106 has a controller 70 
for controlling the operations and functions of each of 

15 the circuits described above. The controller 70 receives 
an Instruction on the GOP structure from the operator or 
the host computer to determine the picture type of each 
picture in a fashion corresponding to the specified GOP 
structure. The controller 70 also receives information on 

20 a target bit rate from the operator or the host computer 
to control the quantization circuit 57 so that a bit rate 
output from the encoding device 106 equals the speci- 
fied one. 

[0131] Further, the controller 70 receives history 
25 information for a plurality of generations output from the 
history information-separating device 105, to reuse the 
information to encode the reference picture. The details 
will be described below. 

[0132] First the controller 70 determines whether 
30 or not the picture type of the reference picture deter- 
mined from the GOP structure specified by the operator 
is the same as the picture type contained in the history 
information. That is, it determines whether or not the ref- 
erence picture has ever been encoded into the picture 
35 type that is the same as the specified one. 

[0133] For better understanding, the example 
shown in Figure 15 will be referenced. The controller 70 
determines whether or not the picture type assigned to 
the reference picture for the fourth-generation encoding 
40 process is the same as the picture type of the reference 
picture for the first-, second-, or third-generation encod- 
ing process. 

[01 34] If the picture type specified for the reference 
picture for the fourth-generation encoding process is dif- 

45 ferent from all the picture types for the past encoding 
processes, the controller 70 carries out a "normal 
encoding process." That is, in this case, this reference 
-picture has not been encoded, during the first-, second- 
, or third-generation encoding process, into the picture 

so type assigned for the fourth-generation encoding proc- 
ess. On the other hand, If the picture type specified for 
the reference picture for the fourth-generation encoding 
process is the same as one of the picture types for the 
past encoding processes, then the controller 70 exe- 

55 cutes a "parameter-reused encoding process." That is, 
in this case, this reference picture has been encoded, 
during the first-, second-, or third-generation encoding 
process, into the picture type assigned for the fourth- 
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generation encoding process. 

[0135] First, the normal encoding process of the 
controller 70 will be described. 

[0136] In order to determine whether to select the 
frame or field prediction mode, the motion vector-detect- 
ing circuit 50 detects a predicted error both in the frame 
prediction mode and in the field prediction mode and 
supplies the difference between the prediction errors to 
the controller 70. The controller 70 compares the pre- 
dicted error values together to select the prediction 
mode with the smaller value. The prediction mode- 
switching circuit 52 processes signals depending on the 
prediction mode selected by the controller 70 to supply 
the processed signals to the calculator 53. 
[0137] Specifically, if the frame prediction mode has 
been selected, the prediction mode-switching circuit 52 
processes an input brightness signal so that the signal 
is output to the calculator 53 as it is, while processing an 
input color difference signal so that the signal has odd- 
and even-number field lines mixed therein, as described 
with reference to Figure 8. On the other hand, If the field 
prediction mode has been selected, the prediction 
mode-switching circuit 52 processes the brightness sig- 
nal In a manner such that the brightness blocks Y[1] and 
Y[2] are configured by odd-number field lines, whereas 
the brightness blocks Y[3] and Y[4] are configured by 
even-number field lines, as described with reference to 
Figure 9. The prediction mode-switching circuit 52 also 
processes the color difference signal in a manner such 
that the upper four lines are configured by odd-number 
field lines, whereas the lower four lines are configured 
by even-number field lines, also as described with refer- 
ence to Figure 9. 

[0138] Furthermore, to select one of the intraimage 
prediction mode, the forward prediction mode, the back- 
ward prediction mode, and the bidirectional prediction 
mode, the motion vector-detecting circuit 50 generates 
and supplies prediction errors in these prediction modes 
to the controller 70. The controller 70 selects the small- 
est of the forward, backward, and bidirectional predic- 
tion errors as an interprediction predicted error. Further, 
it compares this interprediction predicted error with the 
intraimage predicted error to select the smaller value in 
order to select as a prediction mode a mode corre- 
sponding to the selected predicted error. That is, if the 
intraimage predicted error is smaller, the intraimage 
prediction mode is set. If the interprediction predicted 
error is smaller, one of the forward, backward, and bidi- 
rectional prediction modes is set which has the smallest 
predicted error. The controller 70 controls the calculator 
53 and the motion compensating circuit 64 in a fashion 
corresponding to the selected prediction mode. 
[0139] To select either the frame or field DCT mode, 
the DCT mode-switching circuit 55 converts the data in 
the four brightness blocks into a signal form with odd- 
and even-number field lines mixed therein (the frame 
DCT mode), while converting the same data into a sig- 
nal form with these two types of field lines separated 



from each other (the field DCT mode), the DCT mode- 
switching circuit 55 then delivers these signals to the 
DCT circuit 56. The DCT circuit 56 calculates the 
encoding efficiency of a DCT process with odd- and 

5 even-number fields mixed therein and the encoding effi- 
ciency of a DCT process with these two types of fields 
separated from each other and delivers the result to the 
controller 70. The controller 70 compares the encoding 
efficiencies delivered from the DCT circuit 56 to select 

10 one of the DCT modes which has the higher encoding 
efficiency in order to control the DCT mode-switching 
circuit 55 to enter the selected DCT mode. 
[0140] The controller 70 receives a target bit rate 
supplied from the operator or the host computer and a 

15 signal indicative of the amount of bits buffered in the 
transmit buffer 59, that Is, a signal indicative of the 
amount of remaining buffer, and based on the target bit 
rate and the amount of remaining buffer, generates 
feedback_q_scale_code for controlling a quantization 

20 step size for the quantization circuit 57. The 
feedback_q_scale_code is a control signal generated 
depending on the amount of buffer remaining in the 
transmit buffer 59, to preclude its overflow or underflow. 
This control signal also controls the bit rate of a bit 

25 stream output from the transmit buffer 59, to a target 
value. 

[0141] Specifically, if, for example, the amount of 
bits buffered in the transmit buffer 59 decreases, the 
quantization step size is reduced to increase the 

30 amount of bits generated for the next picture to be 
encoded. On the other hand, if the amount of bits buff- 
ered in the transmit buffer 59 increases, the quantiza- 
tion step size is increased to reduce the amount of bits 
generated for the next picture to be encoded. The quan- 

35 tization step size is proportional to the 
feedback_q_scale_code, and increases and decreases 
therewith. 

[01 42] Next, a parameter-reused encoding process, 
which is a characteristic of the transcoder 101, will be 

40 explained. For easier understanding of this process, the 
reference picture is assumed to have been encoded into 
the P picture during the first-generation encoding proc- 
ess, into the I picture during the second-generation 
encoding process, and into the B picture during the 

45 third-generation encoding process, so that it must be 
encoded into the P picture during the fourth-generation 
encoding process. 

[0143] In this case, since the reference picture has 
been encoded, during the first-generation encoding 

so process, into the same picture type (I picture) as that 
assigned as the fourth-generation picture type, the con- 
troller 70 carries out an encoding process using first- 
generation encoding parameters without generating 
new encoding parameters from supplied video data. 

55 The encoding parameters reused for the fourth-genera- 
tion encoding process typically include 
quantiser_scale__code indicative of the quantization 
scale step size, macroblock_type indicative of the pre- 
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diction direction mode, motion_vector indicative of the 
motion vector, frame/field_motion type indicative of 
either the frame or field prediction mode, and dct_type 
Indicative of either the frame or field DCT mode. 
[0144] The controller 70 does not reuse all the 
encoding parameters transmitted as the history infor- 
mation but reuses those encoding parameters that are 
assumed to be desirably reused as described above, 
while newly generating those encoding parameters that 
are not assumed to be desirably reused. 
[0145] Next, this encoding parameter-reused 
encoding process will be explained by focusing on dif- 
ferences from the above described normal encoding 
process. 

[0146] The motion vector-detecting circuit 50 
detects the motion vector of the reference picture during 
the above described normal encoding process, but dur- 
ing this parameter-reused encoding process, reuses the 
motion vector motion_yector supplied as the first-gener- 
ation history information without a detection process. 
The reason will be described below. 
[0147] The baseband video data obtained by 
decoding the third-generation encoded stream has 
been subjected to decoding and encoding processes at 
least three times, so that it evidently has a lower image 
quality than the original video data. An accurate motion 
vector cannot be detected from video data with a 
reduced image quality. That is, the motion vector sup- 
plied as the first-generation history information is defi- 
nitely more accurate than that detected during the 
fourth-generation encoding process. That is, by reusing 
the motion vector supplied as the first-generation 
encoding parameters, the fourth-generation encoding 
process is prevented from degrading the image quality. 
The controller 70 delivers the motion vector 
motion_vector supplied as the first-generation history 
information, to the motion compensating circuit 64 and 
the variable-length encoding circuit 58 as motion vector 
information for the reference picture, which is encoded 
during the fourth-generation encoding process. 
[0148] Further, to determine whether to select the 
frame or field prediction mode, the motion vector-detect- 
ing circuit 50 detects the prediction error both in the 
frame prediction mode and in the field prediction mode. 
During this parameter-reused encoding process, how- 
ever, the motion vector-detecting circuit 50 reuses the 
frame/fie ld_motion_type indicative of either the frame or 
field prediction mode and supplied as the first-genera- 
tion history information, without detecting the prediction 
error both in the frame prediction mode and in the field 
prediction mode. This is because the prediction error in 
each prediction mode detected in the first generation is 
more accurate than that detected during the fourth-gen- 
eration encoding process and because selection of the 
prediction mode determined based on the more accu- 
rate prediction error enables a more optimal encoding 
process. 

[0149] Specifically, the controller 70 supplies the 



prediction mode-switching circuit 52 with a control sig- 
nal, corresponding to the frame/fie ld_motion_type deliv- 
ered as the first-generation history information, and the 
prediction mode-switching circuit 52 executes a signal- 
5 ing process corresponding to the reused 
frame/fieId_motion_type. 

[0150] Further, during the normal encoding proc- 
ess, the motion vector-detecting circuit 50 calculates 
the prediction error in each prediction direction mode in 

io order to determine which to select from the intralmage 
prediction mode, the forward prediction mode, the back- 
ward prediction mode, and the bidirectional prediction 
mode (this prediction mode Is hereafter referred to as a 
■prediction direction mode"). During this parameter- 

75 reused encoding process, however, the motion vector- 
detecting circuit 50 determines the prediction direction 
mode based on the macroblockjype supplied as the 
first-generation history information without calculating 
the prediction error in each prediction direction mode. 

20 This is because the prediction error in each prediction 
direction mode during the first-generation encoding 
process is more accurate than that during the fourth- 
generation encoding process and because selection of 
the prediction direction mode determined based on the 

25 more accurate prediction error enables a more efficient 
encoding process. Specifically, the controller 70 selects 
the prediction direction mode indicated by the 
macroblock_type contained in the first-generation his- 
tory information, and then controls the calculator 53 and 

30 the motion compensating circuit 64 in a fashion corre- 
sponding to the selected prediction direction mode. 
[0151] During the normal encoding process, the 
DCT mode-switching circuit 55 supplies the DCT circuit 
56 with both converted signals in the frame and field 

35 DCT mode signal forms in order to compare the encod- 
ing efficiencies in the frame and field DCT modes. Dur- 
ing this parameter-reused encoding process, however, 
the DCT mode-switching circuit 55 only carries out 
processing corresponding to the DCT mode indicated 

40 by the dct_type contained In the first-generation history 
information without generating both converted signals in 
the frame and field DCT mode signal forms. Specifically, 
the controller 70 reuses the dct_type contained in the 
first-generation history information to control the DCT 

45 mode-switching circuit 55 so as to execute a signaling 
process corresponding to the DCT mode indicated by 
the dctjype. 

[0152] During the normal encoding process, the 
controller 70 controls the quantization step size for the 

so quantization circuit 57 based on the target bit rate spec- 
ified by the operator and the amount of remaining trans- 
mit buffer. During this parameter-reused encoding 
process, the controller 70 controls the quantization step 
size for the quantization circuit 57 based on the target 

55 bit rate, the amount of remaining transmit buffer and the 
past quantization scale contained in the history Informa-. 
tion. In the following description, the past quantization 
scale contained in the history information is described 
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as history_q_seale_code. In a history stream, described 
later, this quantization scale is also described as 
quantiser_scale_code. 

[0153] First, the controller 70 generates the current 
quantization scale feedbacK_q_scale_code as in the 
normal encoding process. The feedback_q_scale_code 
is a quantity determined depending on the amount of 
buffer remaining In the transmit buffer 59 to prevent its 
overflow and underflow. Subsequently, the controller 70 
compares the past quantization scale 
history_q_scale_code contained in the first-generation 
history stream with the current quantization scale 
f eedback_q_scal e_code to determine the larger one. A 
larger quantization scale means a larger quantization 
step. If the current quantization scale 
feedback_q_scale_code Is larger than the past quanti- 
zation scale history_q_scale_code, the controller 70 
supplies the current quantization scale 
feedback__q_scale_code to the quantization circuit 57. 
On the other hand, if the past quantization scale 
hlstory_q_scale_code is larger than the current quanti- 
zation scale feedback_q_scale_code, the controller 70 
supplies the past quantization scale 
hlstory_q_scale_code to the quantization circuit 57. 
[0154] That is, the controller 70 selects the largest 
quantization scale code from the plurality of past quan- 
tization scales contained in the history information and 
the current quantization scale calculated from the 
amount of remaining transmit buffer. In other words, the 
controller 70 controls the quantization circuit 57 to carry 
out quantization using the largest one of the quantiza- 
tion steps used for the past (first-, second-, and third- 
generation) encoding processes arid for the current 
(fourth-generation) encoding process. The reason will 
be described below. 

[0155] For example, it is assumed that a stream 
generated during the third-generation encoding process 
has a bit rate of 4 Mbps and that a target bit rate of 15 
Mbps is set for the encoding device 1 06 that carries out 
this fourth-generation encoding process. Although the 
target bit rate is higher, an appropriate result is not 
obtained simply by reducing the quantization step. The 
image quality of a picture encoded with a large quanti- 
zation step during the past encoding process cannot be 
improved if it is encoded with a reduced quantization 
step during the current encoding process. That is, 
encoding with a quantization step smaller than one for a 
past encoding process simply increases the amount of 
bits and fails to improve image quality. Thus, the most 
efficient encoding process can be achieved by using for 
quantization the largest one of the quantization steps 
used for the past (first-, second-, and third-generation) 
encoding processes and for the current (fourth-genera- 
tion) encoding process. 

[0156] Next, the history decoding device 104 and 
history encoding device 107 in Figure 15 will further be 
described. Although in Figure 15, the history decoding 
device 104 is shown to be a circuit or device different 



from the decoding device 102, it is simply shown in a 
block different from the one in which the decoding 
device 102 is shown, in order to describe the functions 
and configuration of the history decoding device 104 In 

5 an easy^to-understand manner. Processing by the his- 
tory decoding device 1 04 is actually earned out by a var- 
iable-length decoding circuit and a decoding control 
circuit (a decoding controller) in the decoding device 
102. Similarly, although in Figure 15, the history encod- 

7 0 ing device 1 07 is shown to be a circuit or device different 
from the encoding device 106, it Is simply shown in a 
block different from the one in which the decoding 
device 106 is shown, in order to describe the functions 
and configuration of the history decoding device 107 in 

is an easy-to-understand manner. Processing by the his- 
tory encoding device 1 07 is actually carried out by a var- 
iable-length encoding circuit and an encoding control 
circuit (an encoding controller) in the encoding device 
102. 

20 [0157] As shown in Figure 15, the history decoding 
device 1 04 is configured by a user data decoder 201 for 
decoding user data supplied by the decoding device 
102, a converter 202 for converting an output from the 
user data decoder 201 , and a history VLD 203 for repro- 

25 ducing history information from an output from the con- 
verter 202. 

[0158] In addition, the history encoding device 107 
is configured by a history VLC 21 1 for formatting encod- 
ing parameters for the three generations supplied by the 

30 history information-separating device 105, a converter 
212 for converting an output from the history VLC 211, 
and a user data formatter 213 for formatting an output 
from the converter 21 2 into a user data format. 
[0159] The user data decoder 201 decodes user 

35 data supplied by the decoding device 102 to output the 
converted data to the converter 202. The details will be 
discussed later with reference to Figure 51 . The user 
data (user_data()) consists of user_data_start_code 
and user-data, and the MPEG standard prohibits the 

40 user_data from containing continuous 23 bits of *0* (that 
are identical to the start_code). This is to prevent this 
data from being mistakenly detected as the start_code. 
The history information (history_streamQ) is described 
in the user data area (as a type of user.data according 

45 to the MPEG standard) and may contain such continu- 
ous 23 bits or more of "0," so that '1* must be inserted 
into this information in accordance with a predeter- 
mined timing to convert it Into 
converted_history_jstream() (see Figure 38, described 

so later), in order to prevent the occurrence of 23 bits or 
more of "0/ This conversion is carried out by the con- 
verter 212 of the history encoding device 107. The con- 
verter 202 of the history decoding device 104 carries 
out an inverse conversion process to the converter 212 

55 (that removes th e " 1 " inse rted to prevent th e occu rrence 
of 23 bits or more of •0"). 

[0160] The history VLD 203 generates the history 
information (in this case, the first- and second-genera- 
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tion encoding parameters) from the output from the con- 
verter 202 to output it to the history information 
multiplexing device 1 03. 

[0161] On the other hand, in the history encoding 
device 107, the history VLC 211 converts encoding 
parameters tor the three generations (first, second, and 
third generations) supplied by the history information- 
separating device 1 05, into a history Information format 
This format includes a fixed length type (see Figures 40 
to 46, described later) and a variable-length type (see 
Figure 47 and subsequent figures). The details will be 
described later. 

[0162] The history Information formatted by the his- 
tory VLC211 is converted into the 
converted_history_streamO by the converter 212. This 
is to preclude the start_code from being mistakenly 
detected from the user_data() as described above. That 
is, although the history information contains continuous 
23 bits or more of *0, - since such continuous 23 bits or 
more of "0" cannot be arranged within the user_data, 
the converter 212 converts the data (Inserts "1" Into the 
data in accordance with the predetermined timing) so 
as not to deviate from the prohibition. 
[0163] The user data formatter 213 adds 
History_Data_ID to the converted_history_stream() 
supplied by the converter 212, based on Figure 38, 
described later, and further adds 
user_data_stream_code thereto to generate user_data 
in compliance with the MPEG standard so that this data 
can be inserted into a video stream and then outputs it 
to the decoding device 106: 

[0164] Figure 20 represents an example of a config- 
uration of the history VLC21 1 . Its code word converter 
301 and code length converter 305 are supplied from 
the history information-separating device 105 with 
encoding parameters (that are currently transmitted as 
the history information) (Item data) and Information (for 
example, the name of a syntax (for example, the name 
of sequence_header, described later)) (item No.) identi- 
fying a stream in which these encoding parameters are 
to be arranged. The code word converter 301 converts 
the input encoding parameters into code words corre- 
sponding to an indicated syntax and outputs the 
obtained code words to a barrel shifter 302. The barrel 
shifter 302 shifts the code words input by the code word 
converter 301 by an amount corresponding to a shift 
amount supplied by an address generating circuit 306, 
and outputs the shifted code words to a switch 303 in 
bytes. The switch 303, which is switched by a bit select 
signal output by the address generating circuit 306, is 
identical in number to required bits, and delivers the 
code words supplied by the barrel shifter 302 to a RAM 
304 for storage. Required write addresses are specified 
by the address generating circuit 306. In addition, when 
the address generating circuit 306 specifies readout 
addresses, corresponding data (code words) stored in 
the RAM 304 are read out and supplied to the following 
converter 212, and delivered again to the RAM 304 via 



the switch 303 for storage. 

[0165] The code length converter 305 determines 
the code length of the input encoding parameters from 
the input syntax and these parameters to output it to the 

s address generating circuit 306. The address generating 
circuit 306 generates the above described shift amount, 
bit select signal, or write or readout addresses in corre- 
sponding to the input code length to supply them to the 
barrel shifter 302, the switch 303, or the RAM 304, 

io respectively. 

[0166] As described above, the history VLC211 is 
configured as what is called a variable-length encoder 
to subject Input encoding parameters to variable-length 
encoding before output 

is [0167] Figure 21 represents an example of a config- 
uration of the history VLD 203 for decoding data format- 
ted into the history information as described above. In 
the history VLD 203, encoding parameter data delivered 
by the converter 202 is supplied to and then stored in a 

20 RAM 31 1 . Required write addresses are supplied by an 
address generating circuit 315. The address generating 
circuit 315 also generates readout addresses in accord- 
ance with a predetermined timing to supply them to the 
RAM 31 1 . The RAM 31 1 reads out data stored in the 

25 readout addresses to output it to a barrel shifter 312. 
The barrel shifter 312 shifts the input data by an amount 
corresponding to a shift amount supplied by the address 
generating circuit 315, and outputs the shifted data to 
an inverse code length converter 313 and an inverse 

30 code word converter 314. 

[0168] The inverse code length converter 313 is 
also supplied with the name of a syntax (an item No.) for 
a stream with the encoding parameters arranged 
therein by the converter 202. Based on this syntax, the 

35 inverse code length converter 313 determines a code 
length from the input data (code words) to output it to 
the address generating circuit 315. 
[01 69] Additionally, the inverse code word converter 
314 decodes the data supplied by the barrel shifter 312, 

40 based on the syntax, and outputs the decoded data to 
the history information multiplexing device 103. 
[0170] The inverse code word converter 314 also 
extracts information required to determine what code 
words are contained in the data (that is, information 

45 required to determine how the code words are delim- 
ited) and outputs it to the address generating circuit 
315. Based on this information and the code length 
input by the inverse code length converter 313, the 
address generating circuit 315 generates write and rea- 

so dout addresses to output them to the RAM 31 1 and gen- 
erates a shift amount to output it to the barrel shifter 
312. 

[0171] Figure 22 represents an example of a config- 
uration of the converter 212. In this example, 8-bit data 
55 is read out from a buffer memory 320 located between 
the history VLC211 and the converter 212, at readout 
addresses output by a controller 326, and is then sup- 
plied to a D flip flop (D-FF) 321 for retention. The data 
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read out from the D flip flop 321 is supplied to a stuff cir- 
cuit 323 and to an 8-bit D flip flop 322 for retention. The 
8-bit data read out from the D flip flop 322 and the 8-bit 
data read out from the D flip flop 321 are synthesized 
into 16-bit parallel data, which is then delivered to the 
stuff circuit 323. 

[0172J The stuff circuit 323 inserts the code "1" into 
the data at a position indicated by a signal supplied by 
the controller 326 and indicating the stuff position (stuff- 
ing), and outputs the stuffed data to a barrel shifter 324 
as 1 7-bit data. 

[0173] The barrel shifter 324 shifts the input data 
based on a signal (shift) supplied by the controller 326 
and indicating the shift amount, and extracts 8-bit data 
to output it to an 8-bit D flip flop 325. The data held in the 
D flip flop 325 is then read out therefrom and delivered 
to the following user data formatter 213 via a buffer 
memory 327. At this point, the controller 326 generates 
write addresses with output data to output them to 
buffer memory 327, which is interposed between the 
converter 212 and the user data formatter 213. 
[0174] Figure 23 represents an example of a config- 
uration of the stuff circuit 323. The 16-bit data input by 
the D flip flops 322, 321 is input to a contact a, of each 
switch 331-16 to 331-1. A contact c of a switch 331-i (i = 
0 to 15) is supplied with data for a switch adjacent 
thereto on an MSB side (the upper side of the figure). 
For example, the contact c of the switch 331-12 is sup- 
plied with the 1 3th data from an LSB side supplied to the 
contact a of the switch 331-13 adjacent thereto on the 
MSB side, and the contact c of the switch 331 -13 is sup- 
plied with the 14th data from the LSB side supplied to 
the contact a, of the switch 331-14 adjacent thereto on 
the MSB side. 

[0175] However, the contact a of the switch 331-0 
below the switch 331-1 corresponding to the LSB is 
open. The contact C of the switch 331 -1 6 corresponding 
to the MSB is also open because there is no higher 
switch. 

[0176] A contact b of each switch 331 -0 to 331 -1 6 is 
supplied with the data "1 ." 

[0177] In response to the signal stuff position sup- 
plied by the controller 326 and indicating the position at 
which the data "1 " is inserted, a decoder 332 switches 
one of the switches 331-0 to 331-16 to the contact ^ 
side, thereby switching the LSB-side switch to the con- 
tact c side while switching the MSB-side switch to the 
contact q side. 

[0178] Figure 23 represents an example in which 
the data "1" is inserted into the 13th switch from the LSB 
side. Thus, in this case, the switches 331-0 to 331-12 
are each switched to the contact c side, the switch 331- 
13 is switched to the contact side, and the switches 
331-14 to 331-16 are each switched to the contact a 
side. 

[0179] With the above configuration, the converter 
212 in Figure 22 converts the 22-bit code into 23 bits for 
output. 



[0180] Figure 24 represents timings for output data 
from each section of the converter 212 in Figure 22. 
When the controller 326 of the converter 212 generates 
readout addresses (Figure 24(A)) in synchronism with a 

5 clock in bytes, the corresponding data is read out from 
the buffer memory 320 in bytes and held in the D flip flop 
321 . The data read out from the D flip flop 321 (Figure 
24(B)) is supplied to the stuff circuit 323 and the D flip 
flop 322 for retention. The data held in the D flip flop 322 

70 is further read out (Figure 24(C)) and delivered to the 
stuff circuit 323. 

[0181] Accordingly, first 1-byte data DO is input to 
the stuff circuit 323 (Figure 24(D)) with respect to a tim- 
ing for a readout address A1, 2-bye data composed of 
is the 1 -byte data DO and one byte data D1 is input thereto 
with respect to a timing for the next readout address A2, 
and 2-byte data composed of the data D1 and the data 
D2 is input thereto with respect to a timing for a readout 
address A3. 

20 [01 62] The stuff circuit 323 is supplied with the sig- 
nal stuff position (Figure 24 (E)) by the controller 326, 
indicating the position at which the data "1" is inserted. 
The decoder 332 of the stuff circuit 323 switches to the 
contact £ one of the switches 331-16 to 331-0 which 

25 corresponds to this signal stuff position, thereby switch- 
ing the LSB-side switch to the contact c side while 
switching the MSB-side switch to the contact a side. 
Thus, the data "1" is inserted, so that the staff circuit 
323 outputs data (Figure 24(F)) indicating that the data 

30 "1" has been inserted at the position indicated by the 
signal stuff position. 

[0183] The barrel shifter 324 barrel-shifts the input 
data by an amount indicated by the signal shift (Figure 
24(G)) supplied by the controller 326, and outputs the 
35 shifted data (Figure 24(H)). This output is further held in 
the D flip flop 325 and then output to the following circuit 
(Figure 24 (I)). 

[01 84] The data output by the D flip flop 325 has the 
data "1" inserted after the 22-bit data. Thus, there are 

40 22 continuous zeros between the data "1 ■ and the next 
data "1" even if all the bits between these data are zero. 
[01 85] Figure 25 represents an example of a config- 
uration of the converter 202. The configuration of the 
converter 202, which is configured by a D flip flop 341 to 

45 a controller 346, is essentially the same as that of the 
converter 212, which is comprised the D flip flop 321 to 
the controller 326 but differs therefrom in that a delete 
circuit 343 is interposed in the converter 202 instead of 
the stuff circuit 323 in the converter 21 2. The other part 

so of the configuration is the same as that of the converter 
212 In Figure 22. 

[0186] In the converter 202, the delete circuit 343 
deletes a bit (the data *1 * inserted by the stuff circuit 323 
in Figure 22) in accordance with a signal delete position 
55 output by the controller 346 and indicating the position 
of this bit. 

[01 87] The other operations are the same as in the 
converter 21 2 in Figure 22. 



19 



37 



EP 1 069 779 A1 



38 



[0188] Figure 26 is an example of a configuration of 
the delete circuit 343. In this example, LSB-side 15 bits 
of the 16 bits input by the D flip flops 342, 341 are each 
supplied to a contact a of a corresponding one of 
switches 351 -0 to 351 -14. A contact b of each switch is 
supplied with a bit that is closer to the MSB than the bit 
supplied to the contact a. by one bit A decoder 352 
deletes the bit indicated by the signal delete position 
supplied by the controller 346 and outputs the remain- 
ing 15 bits. 

[0189] Figure 26 represents how the 13th bit from 
the LSB is deleted. Thus, in this case, the switches 351- 
0 to 351 -1 1 are each switched to the contact a side, and 
the 12 bits up to the 12th bit from the LSB are selected 
and output as they are. In addition, the switches 351-12 
to 351-14 are each switched to the contact h side, 
whereby the 14th to 16th data are selected and output 
as the 13th to 15th bit data. 

[01 90] Th e inputs to th e stuff ci rcuit 323 in Figu re 23 
and to the delete circuit 343 in Figure 26 each contain 
16 bits because the 1 6 bits supplied by the D flip flops 

322, 321 are input to the stuff circuit 323 of the con- 
verter 212 in Rgure 22 and because the 16 bits sup- 
plied by the D flip flops 342, 341 are also input to the 
delete circuit 343 of the converter 202 in Rgure 25. In 
Figure 22, the 17 bits output by the stuff circuit 323 are 
barrelrshifted by the barrel shifter 324 to finally select 
and output, for example, 8 bits. Likewise, in the con- 
verter 202 in Rgure 25, the 15 bits output by the delete 
circuit 343 are barrel-shifted by the barrel shifter 344 by 
a predetermined amount to obtain 8 bit data. 
[0191] Rgure 27 represents another example of a 
configuration of the converter 21 2. In this example of a 
configuration, a counter 361 counts the number of con- 
tinuous zeros in the input data and outputs a count 
result to the controller 326. When the counter 361 
counts, for example, 22 continuous 0 bits, the controller 
326 outputs the signal stuff position to the stuff circuit 

323. At the same time, the controller 326 also resets the 
counter 361 to allow it to count contlnuos 0 bits again. 
[0192] The other part of the configuration and the 
other operations are the same as in Rgure 22. 
[0193] Rgure 28 represents another example of a 
configuration of the converter 202. In this example of a 
configuration, a counter 371 counts the number of con- 
tinuous zeros in the input data and outputs a count 
result to the controller 346. When a count value in the 
counter 371 -reaches 22, the controller 346 outputs the- 
signal delete position to the delete circuit 343 and also 
resets the counter 371 to allow it to count continuos 0 
bits again. The other part of the configuration is the 
same as in Figure 25. 

[0194] In this manner, in this example of a configu- 
ration, the data "1" is inserted and deleted as a marker 
bit based on a predetermined pattern (the number of 
continuous data "zeros'). 

[0195] The configurations shown in Figures 27 and 
28 enable more efficient processing than those shown 



in Figures 22 and 25. The code length after conversion, 
however, depends on the source history information. 
[01 96] Rgure 29 represents an example of a config- 
uration of the user data formatter 213. In this example, 
5 when a controller 383 outputs readout addresses to a 
buffer memory (not shown) located between the con- 
verter 212 and the user data formatter 213, data read 
out from the buffer memory is supplied to a contact a 
side of a switch 382 of the user data formatter 213. A 
io ROM 381 stores data such as the user data start code 
and data ID which are required to generate user_dataQ. 
The controller 313 switches the switch 382 to the con- 
tact a side or a contact h. side with respect to a predeter- 
mined timing to select and output as appropriate the 
is data stored in the ROM 381 or the data supplied by the 
converter 212. Thus, the data in the user_data() format 
is output to the encoding device 1 06. 
[0197] Although not shown, the user data decoder 
201 can be implemented by outputting input data via a 
20 switch for deleting inserted data that has been read out 
from the ROM 381 in Rgure 29. 

[01 98] Figure 30 shows how a plurality of transcod- 
ers 101-1 to 101-N are connected in series for use in, 
for example, a video edition studio. A history information 
25 multiplexing device 1 03-i of each transcoder 1 01 -i (i = 1 
to N) writes the latest encoding parameters it has used, 
into that partition of an area for the above described 
encoding parameters in which the oldest encoding 
parameters are recorded. Then, baseband image data 
30 has recorded thereon immediately close four genera- 
tions of encoding parameters (generation history infor- 
mation) corresponding to the same macroblock (Rgure 
18). 

[0199] In an encoding device 106-i of each encod- 
35 ing device 106-i (Rgure 19), the variable-length encod- 
ing circuit 58 encodes video data supplied by the 
quantization circuit 57, based on the current encoding 
parameters delivered by a history information-separat- 
ing device 1 05-i. A bit stream thus generated (for exam- 
40 pie, plcture_headerO) has the current encoding 
parameters multiplexed therein. 

[0200] The variable-length encoding circuit 58 also 
. multiplexes user data (including generation history infor- 
mation) supplied by a history encoding device 107-i, 
as into a bit stream for output (the data is multiplexed into 
the bit stream instead of being embedded therein as 
shown in Rgure 18). The bit stream output by the 
encoding device 106-i Is Input to the following trans- 
coder 101-0+1) via an SDTI (Serial Data Transfer Inter- 
so face). 

[0201] The transcoders 101-1 and 101-0+1) are 
each configured as shown in Figure 15. Thus, process- 
ing by these transcoders is the same as described with 
reference to Figure 15. 
55 [0202] For actual encoding using histories of the 
encoding parameters, if the current I picture is to be 
changed to the P or B type, past histories of the encod- 
ing parameters are examined to search for the past P or 
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B picture, If such a history exists, parameters such as its 
motion vectors are used to change the picture type. On 
the contrary, if no such history is found, changing the 
picture type without motion detection is abandoned. Of 
course, even if no such history is found, the picture type 
can be changed through motion detection. 
[0203] Although, in the format shown in Figure 18, 
four generations of encoding parameters are embedded 
in the data, parameters for each of the I, P, and B picture 
types may be embedded therein. Figure 31 Is an exam- 
ple of a format in this case. In this example, when the 
same macroblock has been encoded while changing 
the picture type, a generation of encoding parameters 
are recorded for each picture type (picture history infor- 
mation). Accordingly, the decoding device 102 shown in 
Figure 16 and the encoding device 106 shown in Rgure 
19 input and output a generation of encoding parame- 
ters corresponding to the I. P, and B pictures instead of 
encoding parameters for the current (latest), third, sec- 
ond, and first generations. 

[0204] Additionally, In this case, empty areas 
Cb[1][x] and Cr[1][x] are not used, so that the present 
invention is applicable to image in a 4:2:0 format free 
from the areas Cb[1][x] and Cr[1][x]. 
[0205] In this case, the decoding device 102 
obtains encoding parameters simultaneously with 
decoding, determines the picture type, writes the 
encoding parameters to a portion of the image signal 
corresponding to the picture type (multiplexing), and 
outputs the signal to the history information-separating 
device 105. The history information-separating device 
105 can separate the encoding parameters from the 
signal to carry out encoding while changing the picture 
type taking into consideration the picture type desired 
for encoding and the input past encoding parameters. 
[0206] Next, a process for allowing each transcoder 
101 to determine possible destination picture types will 
be described with reference to the flowchart in Rgure 
32. The picture type changing by the transcoder 101 
must be carried out without motion detection because it 
uses part motion vectors. In addition, the processing 
described below is executed by the history information- 
separating device 105. 

[0207] At step S1 , a generation of encoding param- 
eters (picture history information) are input to the history 
information-separating device 105 for each picture type. 
[0208] At step S2, the history information-separat- 
ing device 105 determines whether or not the picture 
history information contains encoding parameters used 
for a change to the B picture. If the history information- 
separating device 105 determines that the picture his- 
tory information contains encoding parameters used for 
a change to the B picture, the process proceeds to step 
S3. 

[0209] At step S3, the history information-separat- 
ing device 105 determines whether or not the picture 
history information contains encoding parameters used 
for a change to the P picture. If the history information- 



separating device 105 determines that the picture his- 
tory information contains encoding parameters used for 
a change to the P picture, the process proceeds to step 
S4. 

5 [0210] At step S4, the history Information-separat- 
ing device 105 determines that the possible destination 
picture types are the I, P, and B pictures. 
[021 1] If It Is determined at step S3 that the picture 
history information contains no encoding parameters 

10 used for a change to the P picture, the process 
advances to step S5. 

[0212] At step S5, the history information-separat- 
ing device 105 determines that the possible destination 
picture types are the I and B pictures. Further, the his- 

15 tory information-separating device 105 determines that 
special processing (only the forward prediction vectors 
contained in the B picture history information are used 
without the backward prediction vectors) can be used to 
change to the P picture in a pseudo manner. 

20 [021 3] If it is determined at step S2 that the picture 
history information contains no encoding parameters 
used for a change to the B picture, the process 
advances to step S6. 

[0214] At step S6, the history information-separat- 
es ing device 105 determines whether or not the picture 
history information contains encoding parameters used 
for a change to the P picture. If the history information- 
separating device 105 determines that the picture his- 
tory information contains encoding parameters used for 
30 a change to the P picture, the process proceeds to step 
S7. 

[0215] At step S7, the history information-separat- 
ing device 105 determines that the possible destination 
picture types are the I and P pictures. Further, the his- 
35 tory information-separating device 105 determines that 
special processing (only the forward prediction vectors 
contained in the P picture history information are used) 
can be used to change to the B picture in a pseudo 
manner. 

40 [021 6] If it is determined at step S6 that the picture 
history information contains no encoding parameters 
used for a change to the P picture, the process 
advances to step S8. At step S8, the history informa- 
tion-separating device 105 determines that only the 

45 possible destination picture type is the I picture because 
no motion vector is present (since this is the I picture, it 
can be changed only to the I picture). 
[0217] Afterthe processing at steps S4, S5, S7, and 
S8, at step S9, the history information-separating 

so device 105 displays the possible destination picture 
types on the display device (not shown) for notification 
to the user. 

[0218] Figure 33 shows an example of picture type 
change. To change the picture type, the number of 
55 frames constituting the GOP is changed. That is, in this 
case, a 4-Mbps Long GOP (first generation) composed 
of frames of N = 15 (number of GOP frames N = 15) and 
M = 3 (occurrence cycle of I or P pictures within the 
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GOP M = 3) is converted into a 50-Mbps Short GOP 
(second generation) composed of frames of N = 1 and 
M = 1 , which is converted back into a 4-Mbps Long GOP 
(third generation) composed of frames of N = 15 and M 
= 3. The broken lines in the figure denote boundaries in 
the GOP. 

[0219] tf the picture type is changed from first gen- 
eration to second generation, ail the frames can have 
their picture types changed to the I picture, as is appar- 
ent from the above description of the possible destina- 
tion picture type-determining process. During this 
picture type change, all motion vectors calculated to 
convert a motion Image (Oth generation) Into the first 
generation are stored (remain) in the picture history 
information. Then, if a conversion into a Long GOP (the 
picture type is changed from second generation to third 
generation) is executed again, since the motion vectors 
for each picture type which have been used to convert 
the Oth generation into the first generation are stored, 
these vectors can be reused to again execute a conver- 
sion into a Long GOP while restraining image quality 
degradation. 

[0220] Figure 34 shows another example of picture 
type changing. In this case, a 4-Mbps Long GOP (first 
generation) of N = 14 and M = 2 is converted into an 1 8- 
Mbps Short GOP (second generation) of N = 2 and M = 
2, which is converted into a 50-Mbps Short GOP (third 
generation) of N = 1 and M = 1 , that is, having one 
frame, which is further converted into a 1 -Mbps random 
GOP (fourth generation) having N frames. 
[0221] In this case, the motion vectors for each pic- 
ture type used for conversion from Oth generation to first 
generation are stored until the conversion from third 
generation to fourth generation. Then, as shown in Fig- 
ure 34, despite a complicated change in picture type, 
image quality degradation can be minimized by reusing 
the stored encoding parameters. Furthermore, by effec- 
tively using the quantization scale contained in the 
stored encoding parameters, encoding can be imple- 
mented white restraining image quality degradation. 
[0222] The reuse of the quantization scale will be 
described with reference to Figure 35. Figure 35 shows 
that a predetermined frame is constantly converted into 
the I picture from first generation to fourth generation, 
whereas only the bit rate is changed to 4, 18, or 50 
Mbps. 

[0223] For example, in converting the first genera- 
tion (4 Mbps) into the second_generation (18. Mbps), 
image quality is not improved even by using a finer 
quantization scale for encoding in response to the 
increase in bit rate. This Is because data previously 
quantized with a rougher quantization scale is not 
restored. Thus, using a finer quantization scale for 
encoding in response to the increase in bit rate shown in 
Figure 35 simply results in an increase in information 
but does not improve image quality. Accordingly, more 
efficient encoding is enabled by providing such control 
as maintains the previously roughest (largest) quantiza- 



tion scale. 

[0224] in changing the third generation to the fourth 
generation, the bit rate decreases from 50 to 4 Mbps, 
but in this case, the previously roughest (largest) quan- 
5 tization scale is also maintained. 

[0225] As described above, in changing the bit rate, 
it is very effective to use a past history of quantization 
scale for encoding. 

[0226] This quantization control process will be 
io explained with reference to the flowchart In Figure 36. 
At step S11, the history information-separating device 
105 determines whether or not input picture history 
information contains encoding parameters for a picture 
type to be converted. If the history information-separat- 
is ing device 105 determines that the input picture history 
Information contains encoding parameters for the con- 
verted picture type, the process - proceeds to step S1 2. 
[0227] At step S 1 2, the history information-separat- 
ing device 105 extracts the history_q_scale_code from 
20 the relevant encoding parameters in the picture history 
information. 

[0228] At step S1 3, the history information-separat- 
ing device 105 calculates the feedback_q__scale_code 
based on the amount of remaining buffer fed back from 

25 the transmit buffer 59 to the quantization circuit 57. 

[0229] At step S1 4, the history information-separat- 
ing device 105 determines whether or not the 
history_q_scale_code is larger (rougher) than the 
feedback_q__scale_code. If the history information-sep- 

30 arating device 105 determines that the 
history_q_scale_code is larger than the 
feedback_q_scale_code, the process advances to step 
S15. 

[0230] At step S1 5, the history information-separat- 
35 ing device 1 05 outputs the history_q_scale_code to the 
quantization circuit 57 as a quantization scale. The 
quantization circuit 57 uses history_q_scale_code to 
carry out quantization. 

[0231] At step S1 6, it is determined whether all the 
40 macroblocks contained in the frame have been quan- 
tized. If it is determined that not all the macroblocks 
have not been quantized yet, the process returns to step 
S1 2 to repeat the processing from step S12 to step SI 6 
until all the macroblocks have been quantized. 
45 [0232] If it is determined at step S14 that the 
history_q_scale_code is not larger (finer) than the 
feeback_q_scale_code, the process advances to step 
.'. S17. 

[0233] At step S1 7, the history information-separat- 
so ing device 105 outputs the feedback_q_scale_code to 
the quantization circuit 57 as a quantization scale. The 
quantization circuit 57 uses feedback__q_sca!e_code to 
carry out quantization. 

[0234] If it is determined at step S1 1 that the history 
55 information contains no encoding parameters for the 
converted picture type, the process advances to step 
S18. 

[0235] At step S1 8, the history information-separat- 
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ing device 1 05 calculates the feedback_q_scale_code 
based on the amount of remaining buffer fed back from 
the transmit buffer 59 to the quantization circuit 57. 
[0236] At step S19, the quantization circuit 57 uses 
feedback_q_sca!e_code to carry out quantization. 
[0237] At step S20, it is determined whether or not 
all the macroblocks contained in the frame have been 
quantized. If it is determined that not all the macrob- 
iocks contained in the frame have been quantized, the 
process returns to step S1 8 to the processing from step 
SI 8 to step S20 until all the macroblocks have been 
quantized. 

[0238] Inside the transcoder 1 01 according to this 
embodiment, the decoding and encoding sides are 
loosely coupled together and the encoding parameters 
are multiptxed Into the image data before transmission, 
as described above. As shown in Figure 37, however, 
the decoding device 1 02 and the encoding device 1 06 
may be directly connected (tightly coupled) together. 
[0239] The transcoder 101 as described in Figure 
15 multiplexes the past encoding parameters into the 
baseband video data before transmission in order to 
supply the encoding device 106 with the past encoding 
parameters for the first, second, and third generations. 
However, multiplexing the past encoding parameters 
into the baseband video data is not essential for the 
present invention, but a transmission path (for example, 
the data transfer bus) different from that for the base- 
band video data may be used to transmit the past 
encoding parameters, as shown in Figure 37. 
[0240] That is, the history device 102, history 
decoding device 104, encoding device 106, and history 
encoding device 107 shown in Figure 37 have exactly 
the same functions and configurations as the decoding 
device 102, history decoding device 104, encoding 
device 106, and history encoding device 107 described 
in Figure 15. 

[0241] The variable-length decoding circuit 112 of 
the decoding device 102 extracts the third-generation 
encoding parameters from the sequence layer, GOP 
layer, picture layer, slice layer, and macroblock layer of 
the third-generation encoded stream St (3rd) and sup- 
plies these parameters to the controller 70 for the his- 
tory encoding device 107 and encoding device 106. The 
history encoding device 107 converts the received third- 
generation encoding parameters into the 
converted-history_stream0 so that the parameters can 
.be described in the user data area of the picture layer, 
and supplies the converted_history_stream() to the var- 
iable-length encoding circuit 58 of the encoding device 
106 as user data. 

[0242] Further, the variable-length decoding device 
112 extracts the user data user_data containing the 
first- and second-generation encoding parameters, from 
the user data area of the picture layer of the third-gener- 
ation encoded stream, and supplies this data to the his- 
tory decoding device 104 and the variable-length 
encoding device 58 of the encoding device 106. The 



history decoding device 104 extracts the first- and sec- 
ond-generation encoding parameters from the history 
stream described in the user data area as 
co nve rted_h isto ry_stream (), and supplies these param- 

5 eters to the controller for the encoding device 1 06. 

[0243] The controller 70 for the encoding device 
106 controls the encoding process by the encoding 
device 106 based on the first- and second-generation 
encoding parameters received from the history decod- 

io Ing device 104 and the third-generation encoding 
parameters received from the encoding device 1 02. 
[0244] The variable-length encoding circuit 58 of 
the encoding device 106 receives from the decoding 
device 102 the user data user_data containing the first- 

75 and second-generation encoding parameters, while 
receiving from the history encoding device 107 the user 
data user_data containing the third-generation encod- 
ing parameters, and describes these user data in the 
user data area of the picture layer of the fourth-genera- 

20 tibn encoded stream as history information. 

[0245] Figure 38 represents a syntax for use in 
decoding MPEG video streams. The decoder decodes 
an MPEG bit stream in accordance with this syntax to 
extract a plurality of meaningful data elements from the 

25 bit stream. In the illustrated syntax, which will be 
described below, functions and conditional statements 
are represented by thin characters, while the data ele- 
ments are represented by thick characters. The data 
elements are each described in mnemonics indicating 

30 its name, bit length, type, and transmission order. 

[0246] First, the functions used in the syntax shown 
in Figure 38 will be explained. 

[0247] A next_start_codeO function searches for a 
start code described in the bit stream. In the syntax 

35 shown in Figure 38, a sequence_header() function and 
a sequence_extension() function are sequentially 
arranged after the next_start_code() function, so that 
there is described in this bit stream data elements 
defined by the sequence_header() and 

40 sequence_extension() functions. Consequently, during 
decoding of the bit stream, the next_start_code() func- 
tion finds start codes (a kind of data element) in the bit 
stream, which are described in the heads of the 
sequence_header() and sequence_extension() func- 

45 tions, and uses these start codes as references to find 
the sequence_headerQ and sequence_extension() 
— functions. The next_start_codeQ function then decodes 
the data elements defined by the sequence Jieader() 
and sequence_extension() functions. 

so [0248] The sequence_header function defines 
header data for the sequence layer of the MPEG bit 
stream, and the sequence_extension() function defines 
extension data for the sequence layer of the MPEG bit 
stream. 

55 [0249] A do{}while syntax following the 
sequence_extension() function extracts from the data 
stream data elements described based on a function in 
the Q of the do statement while a condition defined by 
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the while statement is true. That is, the doQwhile state- 
ment executes a decoding process for extracting from 
the data stream the data elements described based on 
the function in do statement, while the condition defined 
by the while statement is true. 

[0250] A nextbits() function used in the while state- 
ment compares bits or a bit string appearing in the bit 
stream with the next data element to be decoded. In the 
example of syntax in Figure 38, the nextbitsQ function 
compares the bit string In the bit stream with 
sequence_end_code indicative of the end of the video 
sequence, and when the bit string in the bit stream and 
the sequence_end_code do not match, the condition 
indicated by the while statement is true. Accordingly, the 
doQwhile syntax following the sequence_extension() 
function Indicates that the data elements defined by the 
function in the do statement is described in the bit 
stream unless the seqence_end_code indicative of the 
end of the video sequence appears. 
[0251] In the bit stream, the data elements defined 
by the sequence_extenslon() function are followed by 
data elements defined by extensk>n_and_user_data(0) 
function. The extension_and_user_data(0) function 
defines extension and user data for the sequence layer 
of the MPEG bit stream. 

[0252] A do Qwhile syntax following the 
extension_and_user_data(0) function extracts from the 
bit stream data elements described based on a function 
In the 0 of the do statement while a condition defined by 
the while statement is true. A nextbits() function used in 
the while statement determines whether bits or a bit 
string appearing in the bit stream match(es) 
picture_start_code or group_start_code. If the bits or bit 
string appearing in the bit stream match(es) the 
picture_start_code or group_start_code, the condition 
defined by the while statement is true. Thus, when the 
plcture_start_code or group__start_code appears in the 
bit stream, since this start code describes codes for the 
data elements defined by the function in the do state- 
ment, the doQwhile syntax can search for the start code 
indicated by the picture_start__code or 
group_start_code to extract from the bit stream the data 
elements defined in the do statement. 
[0253] The if statement described at the beginning 
of the do statement indicates the condition that the 
group_start_code appears in the bit stream. If the con- 
dition indicated by this if statement is true, then in the 
description of this bit stream, the group_starrcode is 
followed by data elements defined by a 
group_of_picture_header(1) function and an 
extension_and_user_data(1) function. 
[0254] The group_of_picture_header(1) function 
defines header data for the GOP layer of the MPEG 
data stream. 

[0255] The extension_and_user_data(1) function 
defines extension data (extension_data) and user data 
(user_data) for the GOP layer of the MPEG bit stream. 
[0256] Further, in the description of this bit stream, 



the data elements defined by the 
group_of_picture_header(1) and 
extension_and_user_data(1) functions are followed by 
data elements defined by a plctureJieaderO function 

5 and a picture_coding_extension0 function. Of course, if 
the condition indicated by the if statement described 
above is not true, the data elements defined by the 
group_of_picture^header(1) and 
extension_and_user_data(1) functions are not 

jo described, so that the data elements defined by the 
extension_and_user_data(0) function are followed by 
the data elements defined by the plcture_header() and 
pfcture_codirig_extensionO functions. 
[0257] The picture_header() function defines 

is header data for the picture layer of the MPEG bit 
stream. 

[0258] The picture_coding_extension() function 
defines first extension data for the picture layer of the 
MPEG bit stream. 

20 [0259] The next while statement determines 
whether or not a condition indicated by the next if state- 
ment is true while a condition defined by this while state- 
ment is true. A nextbits() function used in this while 
statement determines whether a bit string appearing in 

25 the bit stream matches extension_start_code or 
user_data_start_code. If the bit string appearing in the 
bit stream matches the extension_start_code or 
user_data_start_code, the condition defined by this 
• while statement is true. 

30 [0260] A first if statement determines whether or 
not the bit string appearing in the bit stream matches 
the extension_start_code. If the bit string appearing in 
the bit stream matches the 32-bit extension_start_code, 
then in the description of the bit stream, the 

35 extension_start_code is followed by data elements 
defined by extension_data(2) function. 
[0261] A second if statement determines whether 
or not the bit string appearing in the bit stream matches 
the user_data_start__code. If the bit string appearing in 

40 the bit stream matches the 32-bit 
user_data_start_^code, it is determined whether or not a 
condition indicated by a third if statement is true. The 
user_data_start_code is a start code indicating the start 
of the user data area of the picture layer of the MPEG bit 

45 stream. 

[0262] The third if statement is a syntax for deter- 
mining whether or not a bit string appearing in the bit 
stream matches History_Data_ID. If the bit string 
appearing in the bit stream matches the 32-bit 

so History_Data_ID, then in the description of the user 
data area of the picture layer of this MPEG bit stream, a 
code indicated by this 32-bit HistoryJDataJD is fol- 
lowed by data elements defined by a 
converted_history_stream() function. 

55 [0263] The converted_history_stream() function 
describes history information and data to transmit all the 
encoding parameters used for MPEG encoding. The 
details of data elements defined by this 
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converted_history_stream() function will be discussed 
below with reference to Figures 40 to 47 as 
history__stream(). In addition, the History_Data_ID is a 
start code indicative of the head of the history informa- 
tion and data described in the user data area of the pic- 
ture layer of the MPEG bit stream. 
[0264] An else statement is a syntax indicating that 
the condition indicated by the third if statement Is not 
true. Accordingly, if the description of the user data area 
of the picture layer of this MPEG bit stream does not 
contain the data elements defined by the 
converted^history_stream() function, it contains the 
data elements defined by the user_data() function. 
[0265] in Figure 38, the history information is 
described in the converted_history_stream() and not in 
the user_data(), whereas the 

converted_history_stream() is described as a kind of 
user_data according to the MPEG standard. Thus, 
although the specification may state that the history 
information is described in the user_data, this means 
that the information Is described as a kind of user_data 
according to the MPEG standard. 
[0266] The picture_data() function describes data 
elements for the slice and macroblock layers after the 
user data for the picture layer of the MPEG bit stream. 
The data elements indicated by the picture_data() func- 
tion are typically described after the data elements 
defined by the converted_history_stream() function 
described in the user data area of the picture layer of 
the bit stream or by the user_data() function. If, how- 
ever, a bit stream indicating the data elements of the 
picture layer does not contain the extension_start_code 
or the user_data_start_code, the data elements indi- 
cated by the picture_data() function are described after 
the data elements defined by the 
picture_coding_extension() function. 
[0267] The data elements indicated by the 
picture_data() function are followed by the data ele- 
ments defined by the sequence header() and 
sequence_extenslon() functions. The data elements 
defined by the sequence_header() and 
sequence_extension() functions are identical to the 
data elements described by the sequence_header() 
and sequence_extension() functions in the head of the 
video stream sequence. The identical data are 
described in the stream to prevent the following error if 
—a bit stream reception device starts receiving the middle 
oi the data stream (for example, a portion of the bit 
stream corresponding to the picture layer), it cannot 
receive the data for the sequence layer, resulting in a 
failure to decode the stream. 

[0268] After the data elements defined by the final 
sequence_header() and sequence_extension() func- 
tions, that is, at the end of the data stream, is described 
the 32-bit sequence„end_code indicative of the end of 
the sequence. 

[0269] A basic configuration for the above syntax is 
schematically shown in Figure 39. 
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[0270] Next, the history stream defined by the 
co nve rted_h isto ry_stream () function will be explained. 
[0271] The converted_history_stream() is a func- 
tion for inserting the history stream indicative of the his- 

5 tory Information Into the user data area of the picture 
layer of the MPEG stream. The term "converted" means 
that this is a stream that has undergone a conversion 
process for inserting a marker bit (1 bit) Into a history 
stream at intervals of at least 22 bits in order to prevent 

to start emulation, the history stream being composed of 
history data to be inserted into the user area. 
[0272] The converted_history_stream0 is 
described in the form of a fixed-length history stream 
(Figures 40 to 46) or a variable length history stream 

is (Figure 47), which will be described below. If the 
encoder selects the fixed-length history stream, the 
decoder can advantageously use simple circuits and 
software to decode each data element from the history 
stream. On the other hand, if the encoder selects the 

20 variable-length history stream, it can arbitrarily select 
the history information (data elements) described in the 
user area of the picture layer to reduce the amount of 
history stream data and thus the data rate of the overall 
encoded bit stream. 

25 [0273] The terms "history stream," "history informa- 
tion," "history data," and "history parameters," as 
described herein, mean the encoding parameters (or 
data elements) used for past encoding processes and 
not the encoding parameters used for the current (final) 

30 encoding process. An example will be described below 
where during the first-generation encoding process, a 
picture is encoded into the I type before transmission, 
where during the second-generation encoding process, 
this I picture is encoded into the P type before transmis- 

35 sion, and where during the third-generation encoding 
process, this P picture is further encoded into the B type 
before transmission. 

[0274] Encoding parameters used for the third-gen- 
eration encoding process are described at predeter- 

40 mined positions In the sequence, GOP, picture, slice, 
and macroblock layers of the encoded bit stream gener- 
ated during the third-generation encoding process. On 
the other hand, encoding parameters used for the first- 
and second-generation encoding processes, past 

45 encoding processes, are not described in the sequence 
or GOP layer in which the encoding parameters used for 
the- third-generation -encoding process- are described 
but in the user data area of the picture layer as history 
information for encoding parameters in accordance with 

50 the above described syntax. 

[0275] First, the fixed-length history stream syntax 
will be described with reference to Figures 40 to 46. 
[0276] Encoding parameters contained in the 
sequence header of the sequence layer used for the 

55 past encoding process (for example, the first- or sec- 
ond-generation encoding process) are first inserted, as 
a history stream, into the user data area of the picture 
layer of a bit stream generated during the final encoding 
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process (for example, the third-generation encoding 
process). It should be noted that history information 
such as the sequence header of the sequence layer of 
a bit stream generated during the past encoding proc- 
ess is not inserted into the sequence header of the 
sequence layer of the bit stream generated during the 
final encoding process. 

[0277] Data elements contained In the sequence 
header (sequence_header) used for the past encoding 
process are composed of sequence_header_code, 
sequenced eader_present_flag, horizontai_size_vaIue, 
marker_bit i vertical_size_va!ue i 
aspect_ratioJnformatlon, frame_rate_code, 
bit_rate_value, VBV_buffer_size_value, 
const rained_parameter_flag, 
load Jntra_q uantiserjnatrix, 
load_nonJntra_quantizer_matrix f 
intra_quantiser_rnatrix, and 
non_intra_quantlser_matrix, etc. 
[0278] The sequence_header_code is data repre- 
senting a start synchronization code for the sequence 
layer. The sequencejieader _present_flag is data indi- 
cating whether or not the data in the sequencejieader 
is valid. The horizontaLslze_value is data consisting of 
lower 12 bits of the number of pixels in a horizontal 
direction of an image. The marker_bit is bit data 
inserted to start code emulation. The 
vertical_size_yalue is data consisting of lower 12 bits of 
the number of vertical lines in an image. The 
aspect_ratio_information is data representing an image 
aspect ratio or a display screen aspect ratio. The 
frame_rate_code is data representing an image display 
cycle. 

[0279] The bit^rate_value is lower-1 8-bit (round-up 
using 400 bsp as a unit) data of the bit rate for limiting 
the amount of generated bits. The 
VBV_buffer_size_value is lower-1 0-bit data of a value 
determining the size of a virtual buffer (a video buffer 
verifier) for controlling the amount of generated codes. 
The constrain ed_parameter_f lag is data indicating that 
each parameter is under the limit. The 
load_intra_iquantiser_matrix is data indicating the pres- 
ence of intra-MB quantization matrix data. The 
load_nonJntra_quantiser_matirx is data indicating the 
presence of non_Jntra-MB quantization matrix data. The 
intra_quantiser_matrix is data indicating a value for the 
—intra-MB ■- -quantization matrix data^ The - non- 
Intra_quantiser_matrix is data indicating a value for non- 
intra-MB quantization matrix data. 
[0280] Following these data elements, data ele- 
ments representing a sequence extension of the 
sequence layer used for the past encoding process are 
described, as a history stream, In the user data area of 
the picture layer of the bit stream generated during the 
final encoding process. 

[0281] The data elements representing the 
sequence extension (sequence_extension) used for the 
past encoding process are extension_start_code, 



extensiOn_start_code_identifier, 
sequence_extension _present_flag, 
p rofi!e_ajid Jeveljncfication, progress"rve_seq u e nee, 
chroma_format, horfzontal_size_extenslon, 

5 verticaLsize_extension f bit_rate_extension, 
vbv_buffer_size__extension, low_delay, 
frame_rate_extenslon_n, frame_rate_extension_d, etc. 
[0282] The extension_start_code ts data represent- 
ing a start synchronization code for the extension data. 

io The extensJon_starLcodeJdentJfler Is data Indicating 
which extension data is sent The 
sequence_extension_present_flag is data indicating 
whether or not the data in the sequence extension Is 
valid. The profile_and_leveUndication is data specify- 

15 ing the profile and level of video data. The 
progressive_sequence Is data indicating that the video 
data is sequentially scanned. The chroma_fbnmat is 
data specifying the chromatic aberration format of the 
video data. 

20 [0283] The horizontal_size_extension is higher-2- 
bit data added to the horizontaLsize_yalue of the 
sequence header. The vertical_size_extension is 
higher-2-bit data added to the vertical_size_value of the 
sequence header. The bit_rate_extension is hlgher-12- 

25 bit data added to the btt_rate_jva!ue of the sequence 
header. The vbv_buffer_size_extension is higher-8-bit 
data added to the vbv_buffer_size_value of the 
sequence header. The low_delay is data indicative of 
the absence of the B picture. The 

30 frame_rate_extension_n is data that is combined with 
the frame_rate_code of the sequence header to obtain 
a frame rate. The frame__rate_extension_d is data that is 
combined with the frame_rate_code of the sequence 
header to obtain the frame rate. 

35 [0284] Following these data elements, data ele- 
ments representing a sequence display extension of the 
sequence layer used for the past encoding process is 
described, as a history stream, in the user area of the 
picture layer of the bit stream. 

40 [0285] The data elements described as the 
sequence display extension 

(sequence_display_extension) are composed of 
extension_start_code, extension_start_code_identrfier, 
sequence_display_extension_present_flag, 

45 video_format, colour_description, colourjirimaries, 
transfer_characteristics, matrix_coefficients, 
-display-horizontal_size, and disp!ay_vertical-size. 
. . [0286]. The extension_starUcode is data represent- 
ing a start synchronization code for the extension data. 

so The extension_start_code_identifier is a code indicating 
which extension data Is sent. The 
sequence_display_extension_present__flag is data indi- 
cating whether or not the data elements in the sequence 
display extension are valid. The video_format is data 

55 representing the video format of a source signal. The 
colour_description is data Indicative of the presence of 
detailed data on a color space. The colour_primaries is 
data indicative of the details of the color characteristics 
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of the source signal. The t ra nsf e r_ch aract e ristics is data 
indicating in detail how photoelectric conversion has 
been carried out. The matrix_co efficients is data indi- 
cating in detail how the primary colors have been con- 
verted into the source signal. The 
display_horizontal_size is data representing an active 
area (horizontal size) of an intended display. The 
display_verticat_size is data representing an active area 
(vertical size) of an intended display. 
[0287] Following these data elements, a macrob- 
lock assignment data 

(macroblock_assignmentJn_user_data) indicating 
phase information for a macroblock generated during 
the past encoding process Is described, as a history 
stream, in the user area of the picture layer of the bit 
stream generated during the final encoding process. 
[0288] The macroblock_assignmentJn_user_data 
indicating the phase information for the macroblock is 
composed of data elements such as 
macroblock_assigninentj>resent_f!ag, v__phase, and 
h_phase. 

[0289] The macroblock_assignment _present_flag 
is data indicating whether or not the data elements in 
the macroblock_assignment_in_user_data Is valid. The 
v_phase is data indicating vertical phase information for 
use in cutting out a macroblock from the image data. 
The h_phase is data indicating horizontal phase infor- 
mation for use in cutting out a macroblock from the 
image data. 

[0290] Following these data elements, data ele- 
ments representing the GOP header of the GOP layer 
used for the past encoding process is described, as a 
history stream, in the user area of the picture layer of 
the bit stream generated during the final encoding proc- 
ess. 

[0291] The data elements representing the GOP 
header (group_of_picture_header) are composed of 
group_start_code, 

group_of jsicture_header_present_flag, time_code, 
closed_gop, and brokenjink. 

[0292] The group_start_code is data indicating a 
start synchronization code for the GOP layer. The 
g ro up_of _pictu re_h eader_p rese nt_fl ag is data indicat- 
ing whether or not the data elements in the 
group_of_picture_header are valid. The time__code is a 
time code indicating the time from the head of the 

sequence of the leading picture of a GOR The 

closeoLgop is flag data indicating that the image in the 
GOP can be reproduced independently of the other 
GOPs. The brokenjink is flag data indicating that the 
leading B picture in the GOP cannot be reproduced 
accurately due to edition or the like. 
[0293] Following these data elements, data ele- 
ments representing the picture header of the picture 
layer used for the past encoding process is described, 
as a history stream, In the user area of the picture layer 
of the bit stream generated during the final encoding 
process. 



[0294] The data elements for the picture header 
(picture_header) are composed of picture_start_code, 
temporal_reference, picture_coding_type, vby_delay, 
fu ll_pel_f orward_vector, forward_f_code, 

5 njll_pel_backward_vector l and backward_f_code. 

[0295] Specifically, the picture__start_code is data 
representing a start synchronization code for the picture 
layer. The temporal_reference Is data indicating the dis- 
play order of a picture and which is reset at the head of 

10 a GOP. The plcture_coding_type is data indicating the 
picture type. The vbv_delay is data indicating the initial 
state of the virtual buffer during a random access. The 
full_pel_forward_vector is data indicating whether the 
precision of forward motion vectors is in integers or half 

75 pixels. The forward_f_code is data indicating a retrieval 
range of the forward motion vectors. The 
full_pel_backward_vector is data indicating whether the 
precision of backward motion vectors is in integers or 
half pixels. The backward__f_code is data indicating a 

20 retrieval range of the backward motion vectors. 

[0296] Following these data elements, a picture 
coding extension of the picture layer used for the past 
encoding process is described, as a history stream, in 
the user area of the picture layer of the bit stream gen- 

25 erated during the final encoding process. 

[0297] Data elements for the picture coding exten- 
sion (ptoture_codingL_extension) are composed of 
extenston_start_code, extension_start_code_identifier, 
f_code[0][0], f_code[0][1J, f_code[1][0], f_code[1][1], 

30 intra_dc_precision, pictu restructure, top_field_first, 
framejDredictive_frame_dct l 

conceal ment_motion_vectors, q_scale_type, 
intra_vlcjformat, alternate_scan, repeat_first_field, 
chroma_420_type, progress ive_frame, 

35 compos*rte_display_flag, y_axis, field_sequence, 
sub_carrier, burst_amplitude, and sub_canier phase. 
[0298] The extension_start_code is a start code 
indicating the start of extension data for the picture 
layer. The extension_start_code_identifier is a code 

40 indicating which extension cide is sent. The f_code[0][0] 
is data representing a retrieval range of horizontal 
motion vectors in a forward direction. The f_code[0][1 ] is 
data representing a retrieval range of vertical motion 
vectors in the forward direction. The f_code[1][0] is data 

45 representing a retrieval range of horizontal motion vec- 
tors in a backward direction. The f_code[1][1] is data 
representing a retrieval range of vertical motion vectors 
in-the backward direction. 

[0299] The intra_dc precision is data representing 
so the accuracy of a DC coefficient The picture_structure 
Is data indicating whether the picture has a frame or 
field structure. If the picture has the field structure, this 
data also indicates whether the field is higher or lower. 
The top_field_first is data indicating the first field is 
55 higher or lower if the picture has the frame structure. 
The frame_predictive_frame_.dct is data indicating that 
frame mode DCT predictions indicate only the frame 
mode, if the picture has the frame structure. The 
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concealmert_motion__vectors is data indicating that an 
intramacroblock has a motion vector tor concealing 
transmission errors. 

[0300] The q_sca!e_type is data indicating whether 
to use a linear or non-linear quantization scale. The 
intra^vlcjorrnat is data indicating whether another two- 
dimensional VLC is used for the intramacroblock. The 
alternate_scan is data Indicating whether to select a 
zigzag or alternate scan. The repeat_first_fietd is data 
used for 2:3 pull down. The chroma_420_type is data 
representing the same value as the next 
progressive_frame if the signal format is 4:2:0 and oth- 
erwise representing 0. The progress tve_frame Is data 
indicating whether or not this picture has been sequen- 
tially scanned successfully. The composite_display_flag 
Is data indicating whether or not the source signal is 
composite. 

[0301] The v_axis is data used if the source signal 
is PAL The field_sequence is data used if the source 
signal is PAL. The sub_carrier is data used if the source 
signal is PAL The burst_amplitude Is data used If the 
source signal is PAL. The sub_carrier phase is data 
used if the source signal is PAL. 
[0302] Following these data elements, a quantiza- 
tion matrix extension used for the past encoding proc- 
ess is described, as a history stream, in the user area of 
the picture layer of the bit stream generated during the 
final encoding process. 

[0303] Data elements for the quantization matrix 
extension (quant__matrix_extension) are composed of 
exte nsion_start_code, extension_start_code_identif i e r, 
quant_matrix_extension_present__flag, 
load _intra_q uantise r_matrix, 
intra_quantiser_matrix[64], 
load_nonJntra_quantisermatrix, 
non_intra_quantiser_matrix[64], 
load_chromaJntra_q uantise rjm atrlx , 
ch roma_jntra_q uantiser_matrix[64] , 
load_chroma_non_i ntra_quantiser_matrix, and 
chroma_nonjntra_quantlser_matrix[64]. 
[0304] The extension_start_code is a start code 
indicative of the start of this quantization matrix exten- 
sion. The extension_start_code_jdentifier is a code indi- 
cating which extension data is sent. The 
quant_matrix_extension_present_flag is data indicating 
whether or not the data elements in this quantization 
matrix extension are valid. The 
load_intra_quantlserjnatrix is data indicative of the 
presence of quantization matrix data for an intramac- 
roblock. The intra_quantiser_matrix is data indicating a 
value for the quantization matrix for the intramacroblock. 
The load_non_intra_quantiser_matrix is data indicative 
of the presence of quantization matrix data for a non- 
intramacroblock. The non_intra_quantiser_matrix is 
data representing a value for the quantization matrix 
data for the non-intramacroblock. The 
load_chroma_Jntra_q uantise r_matrix is data indicative 
of the presence of quantization matrix data for chro- 



matic aberration intramacroblock. The 
chromaJntra_quantiserjnatrix is data indicative of a 
value for the quantization matrix data for the chromatic 
aberration intramacroblock. The 

5 I o ad_c h roma_n o n_intra_q u a rrtise r_matrix is data indic- 
ative of the presence of quantization matrix data for 
chromatic aberration non-lntramacroblock. The 
chroma_nonJntra_quantiser_matrix is data indicative 
of a value for the quantization matrix data for the chro- 
ma matic aberration non-intramacroblock. 

[0305] Following these data elements, a copyright 
extension used for the past encoding process is 
described, as a history stream, in the user area of the 
picture layer of the bit stream generated during the final 
15 encoding process. 

[0306] Data elements for the copyright extension 
(copyright_extension) are composed of 
extension_start_code, exte nsion_start_code_ide ntifie r, 
copyrighLextension_present_flag, copyright_flag, 
20 copyright_identifier, origin al_or_copy, 

copyright_number_1 , copyright_number_2, and 
copyright_number_3. 

[0307] The extension_start_code is a start code 
indicative of the start of the copyright extension. The 

25 extension_start_code_identifter is a code indicating 
which extension data is sent The 
copyright_extension_present_flag is data indicating 
whether or not the data elements in the copyright exten- 
sion are valid. The copyrightjlag indicates whether or 

30 not a copyright is awarded to coded video data up to the 
next copyright extension or a sequence end. 
[0308] The copyrlghtjdentifier is data identifying a 
copyright registration organization specified by ISO/I EC 
JTC/SC29. The original_or_copy is data indicating 

35 whether data in the bit stream is an original or a copy. 
The copyright_number_1 is data representing bits 44 to 
63 of a copyright number. The copyright_number_2 is 
data representing bits 22 to 43 of a copyright number. 
The copyright_number_3 is data representing bits 0 to 

40 21 of a copyright number. 

[0309] Following these data elements, a picture dis- 
play extension (picture_display_extension) used for the 
past encoding process is described, as a history 
stream, in the user area of the picture layer of the bit 

45 stream generated during the final encoding process. 
[0310] Data elements for this picture display exten- 
sion are composed of extension_start_code, 
extenslon_start_code_identifier, 
picture_displaly_extension_present_flag, 

so frame_center_horizontal_offset_1 , 
frame_center_verticaLoffset_1 , 
frame_center_horizontaLoffset_2, 
frame_center_verticaLoffseL2, 

frame_center_horizontaLoffset_3, and 
55 f rame_ce nte r_verticaLoffset_3. 

[0311] The extension_start_code is a start code 
indicative of the start of the picture display extension. 
The extension_start_codeJdentifiBr is a code indicating 
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which extension data is sent. The 
picture_displaly_extension _present_flag is data indicat- 
ing whether the data elements in the picture display 
extension are valid. The frame_center_horizontal_offset 
is data indicating horizontal offsets in a display area and 
which can define three offset values. The 
frarne_center_vertical_offset is data Indicating vertical 
offsets in the display area and which can define three 
offset values. 

[031 2] The present invention is characterized in that 
the history information representing this picture display 
extension is followed by data elements for 
re_codlng_streamjnformatlon. Data elements for the 
re_codingLstream_information are composed of 
user_data_start_code, re_coding_stream_infoJ D, 
red_bw_flag, redjowjndlcator, or the like. 
[0313] The user_data_start_code is a start code 
indicating that the user_data starts. The 
re_coding_stream_info_ID is a 16-bit integer used to 
identify the re_coding_stream_infor() function. Specifi- 
cally, the value of this integer is "1 001 0001 1 1 1 0 1 1 00" 
(0x91 ec). 

[0314] The red_bw_flag is a 1-bit flag that is 0 for 
transmission of coding parameters for all history infor- 
mation and that is 1 for selective transmission of coding 
parameters for history information. The 
red_bw_indicator is a 2-bit integer that is an indicator for 
defining a data set of coding parameters. 
[0315] The re_codlng_streamJnformatfon, the 
red_bw_flag, the red_bw_indicato, and the data set will 
be described later in detail. 

[0316] User data (user_data) used for the past 
encoding process is described, as a history stream, in 
the user area of the picture layer of the bit stream gen- 
erated during the final encoding process. 
[0317] Following the user data, information on the 
macroblock layer used for the past encoding process is 
described as a history stream. 

[0318] The information on the macroblock layer is 
composed of data elements for the positions of macrob- 
locks (macroblock) such as macrobiock_address__h, 
macroblock_address_v, slice_header_present_fiag, 
and skipped_macrbblock__flag, data elements for a 
macroblock mode (macroblock_modes[)) such as 
macroblock_quant, macroblock u _motion_forward, 
macroblock_motion_backward, macroblock_pattern , 
— macroblockjntra, spatiaLtemporahweight_icode_flag, 
frame_motion^type, and dct_Jype, data elements for 
quantization step control such as 
quantiser_scale_code, data elements for motion com- 
pensation such as PMV[0P][0], PMV[O][0][1], 
motion_verticaljield_select[0][0], PMV[0][1 j[0], 

PMV[0][1][1], motlon_verticaLfield_select[0][1], 
PMV[1][0][0], PMV[1][0][1], 
motion_vertical_field_select[1 ][0], PMV[1 ][1 ][0], 

PMV[1][1][1], and motion_vertfcaljleld_select[1][1], 
data elements for a macroblock pattern such as 
coded_block_pattern, and data elements for the 



amount of generated codes such as numjnv_bits, 
num_coefJbits, and num_other_bits. 
[0319] The data elements for the macroblock layer 
will be described below in detail. 

5 [0320] The macroblocK_address_h Is data for defin- 
ing the horizontal absolute position of the current mac- 
roblock. The macroblock_address_v is data for defining 
the vertical absolute position of the current macroblock. 
The slice_headerj>resent_flag is data indicating 

io whether or not this macroblock is at the head of the slice 
layer and includes a slice header. The 
skipped_macroblock_flag is data indicating whether to 
skip this macroblock during a decoding process. 
[0321] The macroblock_quant is data derived from 

is a macroblock type (macroblock_type), shown in Figures 
63 and 64, described below, and Indicates whether or 
not the quantiser_scale_code appears in the bit stream. 
The macroblock_motion_forward is data derived from 
the macroblock type shown in Figures 63 and 64 and is 

20 used for the decoding process. The 
macroblock_motion_backward Is data derived from the 
macroblock type shown in Figures 63 and 64 and is 
used for the decoding process. The macroblock_pattern 
is data derived from the macroblock type shown in Rg- 

25 u res 63 and 64 and indicates whether or not the 
coded_block_pattern appears in the bit stream. 
[0322] The macroblockjntra is data derived from 
the macroblock type shown in Figures 63 and 64 and is 
used for the decoding process. The 

30 spatiaLtemporaL_weight_code_flag is data derived 
from the macroblock type shown in Figures 63 and 64 
and indicates whether or not the bit stream contains 
spatiaMemporaLweight_code indicative of a method 
for up-sampling lower layer images using time scalabil- 

35 "rty. 

[0323] The frame_motion_type is a 2-bit code indi- 
cating a predicted type of a macroblock in a frame. This 
code is "00" if two prediction vectors are present and are 
of a field-base prediction type, "01 " if one prediction vec- 

40 tor is present and is of the field-base prediction type, 
"1 0" if one prediction vector is present and is of a frame- 
base prediction type, and "1 1" if one prediction vector is 
present and is of a dual prime prediction type. The 
field_motion_type is a 2-bit code indicating a motion 

45 prediction for a macroblock in a field. This code is "01 ■ if 
one prediction vector is present and is of the field-base 
prediction type, "10" if two prediction vectors are™ 
present and are of an 1 8 x 8 macroblock-base predic- 
tion type, and "1 1 ■ if one prediction vector is present and 

so is of a dual prime prediction type. The dct_type is data 
indicating whether the OCT is in a frame or field DCT 
mode. The quantiser_scale_code is data indicating the 
quantization step size of a macroblock. 
[0324] Next, data elements for motion vectors will 

55 be described. A motion vector is encoded into a differ- 
ential from the preceding encoded vector in order to 
reduce the number of motion vectors required for 
decoding. The decoder must maintain four motion vec- 
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tor predicted values (each including horizontal and ver- 
tical components) to decode motion vectors. These 
predicted motion vectors are represented as 
PMV[r][s][v]. The [r] is a flag Indicating a first or a sec- 
ond motion vector in a macroblock, and is "0" for the first 
motion vector and B 1 " for the second motion vector. The 
[s) is a flag indicating whether a motion vector in a mac- 
roblock is a forward or backward vector, and is "CT if this 
motion vector is a forward vector and "1 " if it is a back- 
ward vector. The [v] is a flag indicating whether a hori- 
zontal or vertical vector component in a macroblock, 
and is "0" for a horizontal vector component and "1" for 
a vertical vector component 

[0325] Thus, the PMV[0][0][0] represents data for 
the horizontal component of the forward motion vector 
of the first vector. The PMV[0][0][1] represents data for 
the vertical component of the forward motion vector of 
the first vector. The PMV[0][1][0] represents data for the 
horizontal component of the backward motion vector of 
the first vector. The PM V[0][i ][1 ] represents data for the 
vertical component of the backward motion vector of the 
first vector. 

[0326] The PMV[1][0][0] represents data for the 
horizontal component of the forward motion vector of 
the second vector. The PMV[1][0][1] represents data for 
the vertical component of the forward motion vector of 
the second vector. 

[0327] The PMV[1][1][0] represents data for the 
horizontal component of the backward motion vector of 
the second vector. The PMV[1][1][1] represents data for 
the vertical component of the backward motion vector of 
the second vector. 

[0328] The motion_verticaLfield select[r][sj is data 
indicating which reference field is used for a prediction 
form. If the motion_vector_field_select[r][s] is "0, B a top 
reference field is used. If it is "1 ," a bottom reference 
field Is used. 
[0329] Accordingly, 

motion_vertical_field_select[0][0] indicates a reference 
field for use in generating the forward motion vector of 
the first vector, motion_verticaLfield_select[0][1] indi- 
cates a reference field for use in generating the back- 
ward motion vector of the first vector. 
motion_vertical_field_select[1][0] indicates a reference 
field for use in generating the forward motion vector of 
the second vector, motion_verticaLfield_select(1][1] 
indicates-a reference field for use in generating the 
backward motion vector of the second vector. 
[0330] The coded_block_pattern is data of a varia- 
ble length indicating which of plural DCT blocks that 
store DCT coefficients has a significant coefficient (a 
nonzero coefficient). The num_mv_brts is data indica- 
tive of the amount of codes in a motion vector in a mac- 
roblock. The num_coef_bits is data indicative of the 
amount of codes in a DCT coefficient in a macroblock. 
The num_other_bits is data indicating the amount of 
codes in a macroblock other than the motion vectors 
and DCT coefficients. 



[0331] Next, a syntax for decoding each data ele- 
ment from a history stream of a variable length will be 
explained with reference to Figures 47 to 67. 
[0332] This variable-length history stream is com- 

5 posed of the next_start_codeO function, the 
sequence_header() function, the sequence_extension{) 
function, the extension_and_user„data(0) function, the 
group_of_picture_header() function, the 

extension__and_user_data( 1 ) function, the 

io p!cture_headerO function, the 

picture_coding_extension() function, the 

re_coding_stream_infoO function, the 

extension_anoLuser_data(2) function, and the 
picture_data() function. 

15 [0333] Since the next_start_codeO function 
searches for a start code present in the bit stream, there 
is described in the head of the history stream, data ele- 
ments used for the past encoding process and defined 
by the sequence_header() function as shown in Figure 

20 48. 

[0334] The data elements defined by the 
sequence_header() function is the 

sequence_header_code, the 
sequence_header_present_flag, the 

25 horizontaLsize_value, the vertical_size_value, the 
aspect_ratio_information, the frame_jate_code, the 
bit_rate_value, the marker_bit, the 

VBV_buffer_size_value, the 
constrained_parameterJlag, the 

30 load_intra_quantiser_matrix, intra_quantiser_matrix, 
the load_non_intra_quantiser_matrixm, the 
nonJntra_quantiser_matrix, etc. 

[0335] The sequence_header_code is data repre- 
senting a start synchronization code for the sequence 

35 layer. The sequence_header_present_flag indicates 
whether or not the data in the sequence_header is valid. 
The horizontal_size_value is data consisting of lower 12 
bits of the number of pixels in a horizontal direction of an 
image. The vertical_size_value is data consisting of 

40 lower 1 2 bits of the number of vertical lines In an image. 
The aspect_ratio_information is data representing an 
aspect ratio of the pixel or display screen aspect ratio. 
The frame_rate_code is data representing a image dis- 
play cycle. The b*rt_rate_value is lower-1 8-bit (round-up 

45 using 400 bsp as a unit) data of a bit rate for limiting the 
amount of generated bits. 

[0336] The marker_brt is bit data inserted to prevent 
start code emulation. The VBV_buffer_size_value is 
lower-1 0-b'rt data of a value determining the size of a vir- 

50 tual buffer (a video buffer verifier) for controlling the 
amount of generated codes. The 
constrained_parameter_flag is data indicating that each 
parameter Is under the limit. The 
loadJntra_quantiser_jTiatrix is data indicating the pres- 

55 ence of intra-MB quantization matrix data. The 
lntra_quantiser_matrix Is data Indicating a value for the 
intra-MB quantization matrix data. The 
load_non_intra_quantiser_matrix is data indicating the 
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presence of non_intra-MB quantization matrix data. The 
non-intra_quantiser_matrix is data indicating a value for 
non-intra-MB quantization matrix data. 
[0337] Following the data elements defined by the 
sequence_header() function, data elements defined by 
the sequence_extension()f unction such as those shown 
in Figure 49 are described as a history stream. 
[0338] The data elements defined by the 
sequence_extension() function are the 



extensIon_start_code, the 

extension_start_codeJdentifier, the 

sequence_extension_present__flag, the 

profile_andJeve!Jndlcat!on, the 



progress*rve_sequence, the chroma_format, the 
horizontal_size_extension, the vertical_size_extension, 
the bit_rate_extension, the vbv_buff e r_slze_exte nsio n , 
the low_delay, the frame_rate_jextension_n, the 
frame_rate_extension_d, etc. 

[0339] The extension_start_code is data represent- 
ing a start synchronization code for extension data. The 
extenslon_start_codeJdentif ier is data indicating which 
extension data is sent The 

sequence_extension_present_flag is data indicating 
whether or not the data in the sequence extension is 
valid. The profile_andJeveMndication is data specify- 
ing the profile and level of video data. The 
progressive_sequence is data indicating that the video 
data is sequentially scanned. The chroma_format is 
data specifying the chromatic aberration format of the 
video data. The horizontal_size_extension is higher-2- 
bit data added to the horizontaLsize_value of the 
sequence header. The verticaLsize_extension is 
higher-2-bit data added to the verticaLsize_value of the 
sequence header. The bit_rate_extension is higher-12- 
bit data added to the bit_rate_value of the sequence 
header. The vbv_buffer_size_extension is higher-8-bit 
data added to the vbv_buffer_size__value of the 
sequence header. 

[0340] The low_delay is data indicative of the 
absence of the B picture. The frame_rate_extenslon_n 
is data that is combined with the frame_rate_code of the 
sequence header to obtain a frame rate. The 
frame__rate_extension_d is data that is combined with 
the f rame_rate_code of the sequence header to obtain 
the frame rate. 

[0341] Following the data elements defined by the 
~sequence-extension() function,- data elements defined 
-by the extension = and^user_data(0) function such as 
those shown in Figure 50 are described as a history 
stream. When V is not 1, the 
extension_and_user_data(i) function does not describe 
the data elements defined by the extension_data() func- 
tion but only the data elements defined by the 
user_data() function, as a history stream. 
[0342] The extension_and_user_data(0) function 
describes only the data elements defined by the 
user_data() function, as a history stream. 
[0343] The user_data() function describes the user 



data as a history stream based on such a syntax as 
shown in Figure 51 . 

[0344] Following the data elements defined by the 
extension_and_user_data(0) function, data elements 

5 defined by the group_of_picture_headerO function and 
the extension_and_user_data(1) function, as a history 
stream, as shown in Figure 52. However, the data ele- 
ments defined by the group_of_pteture_header() func- 
tion and the extension_and_user_adta(1) function are 

io described only if the group_start_code indicative of the 
start code for the GOP layer is described in the history 
stream. 

[0345] The data elements defined by the 
group_of_picture_headerO function are composed of 

15 the group_start_code, the 

group_of_plcture_header_present_flag f the time_code, 
the closed gop, and the broken Jink. 
[0346] The group_start__code is data indicating a 
start synchronization code for the GOP layer. The 

20 group_of _picture_header_present_flag is data indicat- 
ing whether or not the data elements In the 
group_of_picture_header are valid. The time_code is a 
time code indicating the time from the head of the 
sequence of the leading picture of a GOP. The 

25 closed_gop is flag data indicating that the image in the 
GOP can be reproduced independently of the other 
GOPs. The broken_link is flag data indicating that the 
leading B picture in the GOP cannot be reproduced 
accurately due to edition or the like. 

30 [0347] Like the extension_and_user_data(0) func- 
tion, the extension_and_user_data(1) function 
describes only the data elements defined by the 
user_data0 function, as a history stream. 
[0348] If the history stream does not contain the 

35 group_start_code indicative of the start code for the 
GOP layer, the history stream has described therein no 
data elements defined by the 
group_of_picture_header() function and the 
extension_and_user_adta(1) function. In this case, fol- 

40 lowing the data elements defined by the 
extension_and_user_data(0) function, data elements 
defined by the picture_header() function are described 
as a history stream. 

[0349] The data elements defined by the 
45 picture.headerO function are the picture_start_code, 
the temporal_reference, the picture_coding_type, the 

vby_delay, the full^pebforward^vector, the 

forward_rcode, the full^peUbackward^vector, the 
backward_f_code, extra_bit _picture, and 
so extraJnformation_picture, as shown in Figure 53. 

[0350] Specifically, plcture_start_code is data rep- 
resenting a start synchronization code for the picture 
layer. The tempo raLreference is data Indicating the dis- 
play order of a picture and which is reset at the head of 
55 a GOP. The picture_coding_type is data indicating the 
picture type. The vbv_delay is data Indicating the initial 
state of the virtual buffer during a random access. The 
full_pel_forward_vector is data indicating whether the 
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precision of forward motion vectors is in integers or half 
pixels. The forward_f_code is data indicating a retrieval 
range of the forward motion vectors. The 
fu!l_pel_backward_vectoris data indicating whether the 
precision of backward motion vectors is in integers or 
half pixels. The backward_f_code is data indicating a 
retrieval range of the backward motion vectors. The 
extra_bit_picture is a flag indicating the presence of 
subsequent additional information. if the 
extra_bitj3icture is "1," the extra Jnfoimation_picture 
follows, whereas if it is "0," no data follows. The 
extrajnfbmiationjaicture Is information reserved for 
the standard. 

[0351] Following the data elements defined by the 
picture_header() function, data elements defined by the 
pIcture_coding_extenslon() function such as those 
shown in Figure 54 are described as a history stream. 
[0352] The data elements defined by the 
picture_coding_extension() function are composed of 
the extension_start_code, the 

extension_start_code_ldentifier, the f_code[0][0], the 
f_code[0][1], the f_code[1][0], the fjsode[l][l], the 
intra_dc_precision, the pkrtu restructure, the 
top_field_first the frame_predictrve_frame__dct, the 
concealment_motion_vectors, the q_scale_type, the 
intra_vlc_format, the atternate_scan, the 
repeat_firt_field, the chroma_420_type, the 
progressive_frame, composite_display_flag, the v_axis, 
the field_sequence, the sub_carrier, the 
burst_ampl"rtude, and the sub_carrier_phase. 
[0353] The extension_start_code is a start code 
indicating the start of extension data for the picture 
layer. The extension_start_code_identifter is a code 
indicating which extension code is sent The 
f_code[0][0] is data representing a retrieval range of 
horizontal motion vectors in a forward direction. The 
f_code[0][1] is data representing a retrieval range of ver- 
tical motion vectors in the forward direction. The 
fjcode[1][0] is data representing a retrieval range of 
horizontal motion vectors in a backward direction. The 
f_code[1 ][1 ] is data representing a retrieval range of ver- 
tical motion vectors in the backward direction. The 
intra_dc_precision is data representing the accuracy of 
a DC coefficient 

[0354] The pictu restructure is data indicating 
whether the picture has a frame or field structure. If the 
-picture has the field structure, this data-also indicates 
whether the field is higher or lower. The top_field_first is 
data indicating whether the first field is higher or lower if 
the picture has the frame structure. The 
frame__predictlve_frame_dct is data indicating that 
frame mode DCT predications indicate only the frame 
mode, if the picture has the frame structure. The 
concealment_motion_vectors is data indicating that an 
intramacroblock has a motion vector for concealing 
transmission errors. The q_scate_type is data indicating 
whether to use a linear or non-linear quantization scale. 
The intra_vlc_format is data indicating whether another 



two-dimensional VLC is used for the intramacroblock. 
[0355] The atternate_scan is data indicating 
whether to select a zigzag or alternate scan. The 
repeat_firt_field is data used for 2:3 pull down. The 

5 chroma_420_type is data representing the same value 
as the next progressive_frame if the signal format is 
4:2:0 and otherwise representing 0. The 
progresslve_frame is data indicating whether or not this 
picture has been sequentially scanned successfully. 

io The composite_dlsplay_flag Is data Indicating whether 
or not the source signal is composite. The v_axis is data 
used if the source signal is PAL. The fie1d_seqijence is 
data used if the source signal Is PAL The sub__carrier Is 
data used if the source signal is PAL. The 

is burst_amplitude is data used if the source signal is PAL. 
The sub_carrier_phase Is data used If the source signal 
is PAL. 

[0356] Following the data elements defined by the 
picture_coding__extensionO function, data elements 

20 defined by the re_coding_streamJnfo() function are 
described as a history stream. The 
re_coding_streamJntbO function is characteristic of the 
present invention and is principally used to describe a 
combination of history information. The details will be 

25 discussed later with reference to Figure 68. 

[0357] Following the data elements defined by the 
re_coding_streamJnfoO function, data elements 
defined by the extensions_and_user_data(2) function 
are described as a history stream. As shown in Figure 

30 50, if the extension start code (extension_start_code) | s 
present in the bit stream contains, the data elements 
defined by the extension_data() function are described 
in the extension_and_user_data(2) function. Following 
these data elements, the data elements defined by the 

35 user_data() function are described if the user data start 
code (user_data_start_code) j s present in the bit 
stream. If the extension start code and the user data 
start code are absent from the bit stream, the data ele- 
ments defined by the extension_data() and user_data() 

40 functions are not described In the bit stream. 

[0358] The extension_data() function describes in 
the bit stream as a history stream, the data elements 
indicating the extension_start_code and data elements 
defined by the quant_matrix_extension(), 

45 copyrighL_extension(), and picture_display_extension() 
functions, as shown in Figure 55. 
[0359] - The data — elements defined by the 
quant_matrix_extension() function are, the 



extension_start_code, me 

so extension_start_code_identifier, the 

quant_matrlx_extension_present_flag, the 

load_intra_quantiser_matrix, the 

intra_qu antise r_matrix[64], 

load_non_intra_quantiser_matrix, 

55 nonJntra_quantiser_matrix[64], the 

load_chromaJntra_quantiser_matrix, the 

ckromaj ntra_q u antise r_matrix[64J, th e 



load_chroma_non_intra_quartiser_matrix, and the 
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chroma_nonJntra_quantiser_matrix[64), as shown in 
Figure 56. 

[0360] The extension_start_code is a start code 
indicative of the start of this quantization matrix exten- 
sion. The extension_start_code_ldentifier Is a code indi- 
cating which extension data is sent The 
quant_matrix_extension_present_flag is data indicating 
whether or not the data elements in this quantization 
matrix extension are valid. The 
loadJntra__quantJser_matrlx is data Indicative of the 
presence of quantization matrix data for ah intramac- 
roblock. The intra_quantiser_matrix is data indicating a 
value for the quantization matrix forthe intramacroblock. 
The load_non_intra_quantiser_matrix is data indicative 
of the presence of quantization matrix data for a non- 
intramacroblock. The nonJntra_quantiser_matrix is 
data representing a value for the quantization matrix 
data for . the non-intramacroblock. The 
load_chromaJntra_quantiser_matrix is data indicative 
of the presence of quantization matrix data for chro- 
matic aberration intramacroblock. The 
chroma_intra_quantiser_matrix is data indicative of a 
value for the quantization matrix data for the chromatic 
aberration intramacroblock. The 

load_chroma_non_intra_quantiser_matrix is data indic- 
ative of the presence of quantization matrix data for 
chromatic aberration non-intramacroblock. The 
chroma_non_intra_quantiser_matrix is data indicative 
of a value for the quantization matrix data for the chro- 
matic aberration non-intramacroblock. 
[0361] The data elements defined by the 
copyright_extension() function are comprised of the 
extension_start_code, the 
extension_start__code_Jdentifier, the 
copyright_extension_present_flag, the copyright_flag, 
the copyrightjdentifier, the origin aLor_copy, the 
copyrigrtt_number_1 , the copyright_number_2, and the 
copyright_number_3, as shown in Figure 57. 
[0362] The extension_start_code is a start code 
Indicative of the start of the copyright extension. The 
extension_start_code_identifier is a code indicating 
which extension data is sent. The 
copyright_extension_present_flag is data indicating 
whether or not the data elements in the copyright exten- 
sion are valid. 

[0363] The copyright_f lag indicates whether or not 
— a copyright is awarded to coded video data up to the 
next copyright extension or a sequence end. The 
copyrightjdentifier is data identifying a copyright regis- 
tration organization specified by ISO/IEC JTC/SC29. 
The original_or_copy Is data indicating whether data in 
the bit stream is an original or a copy. The 
copyrighUnumber_1 is data representing bits 44 to 63 
of a copyright number. The copyright_number_2 is data 
representing bits 22 to 43 of a copyright number. The 
copyright_number_3 is data representing bits 0 to 21 of 
a copyright number. 

[0364] The data elements defined by the 



pirture_display_extension() function are the 
extension_start_code_identifier, the 
frame_center_horizontaLoffset, the 
frame_center_vertical_offset and the like, as shown in 
5 Figure 58. 

[0365] The extension_start_code_identifier is a 
code indicating which extension data is sent The 
frame_center_horizontal_offset is data indicating hori- 
zontal offsets in a display area and which can define a 

io defined umber of offset values using 
number_of_frame_center_offsets. The 
frame_center_verticaLoffset is data incficating vertical 
offsets in the display area and which can define a 
defined number of offset values using 

is number_of_frame_center_offsets. 

[0366] Referring back to Figure 47, following the 
data elements defined by the 

extension_and_user_data(2) function, data elements 
defined by the picture_header() function are described 

20 as a history stream. The picture_data() function, how- 
ever, is present if the red_bw_Jiag is not 1 or if the 
red_bw_indicator is 2 or less. The red_bw_fiag and the 
red_bw_indicator are described in the 
re_coding_stream_infoO function and will be described 

25 later with reference to Figures 68 and 69. 

[0367] The data elements defined by the 
picture_data() function are those defined by the slice() 
function as shown in Figure 59. At least one of these 
data elements defined by the siiceQ function is 

30 described in the bit stream. 

[0368] The slice() function describes as a history 
stream data elements such as the slice_start_code, the 
slice_quantiser_scale__code, the intra_slice__flag, the 
intra_slice, the reserved_b"rts, the extra_bit_slice, the 

35 extra _jnformation_slice, and the extra_bit_slice and 
data elements defined by the macroblock() function. 
[0369] The slice_start_code is a start code indica- 
tive of the start of the data elements defined by the 
slice () function. The slide_quantiser_sca!e_code is a 

40 data indicating a quantization step size set for a mac- 
roblock present in this slice layer. If, however, the 
quantiser_scale_code is set for each macroblock, data 
in macroblock_quantiser_scale code set for each mac- 
roblock is used preferentially. 

45 [0370] The intra__slice_flag is a flag indicating 
whether or not the intra_slice and the reserved_J>its are 
present in the bit stream. The intra_sfice is data indicat- 
ing whether or not a noh -intramacroblock is present in 
the slice layer. If any of the macroblocks in the slice 

so layer is a non-intramacroblock, the intra_slice is "0." If all 
the macroblocks in the slice layer are non-Intramacrob- 
locks, the intra_slice is "1." The reserved_bits is 7-bit 
data and is "0." The extra_bit_slice Is a flag indicative of 
the presence of additional information as a history 

55 stream. If the extra_information_slice follows, this flag is 
set to "1 whereas if no additional information follows, it 
is set to "0." 

[0371] Following these data elements, the data ele- 
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ments defined by the macroblockO function are 
described as a history stream. 

[0372] The macroblock() function describes data 
elements such as macroblock_escape, 
macroblocK_addressJncrement 
macroblock_quant*iser_scaie_code, and marker_b"rt, 
and data elements defined by a macroblock_modes 
function, a motlon_vectors(s) function, and a 
code_block_pattern() function. 
[0373] The macroblocK_escape is a fixed bit string 
indicating whether or not the difference between a refer- 
ence macroblock and the preceding macroblock is 34 or 
more. If the difference between the reference macrob- 
lock and the preceding macroblock is 34 or more, 33 is 
added to the value of the 
macroblock_addressJncrement The 
macroblock_address_increment is data indicating a dif- 
ference in the horizontal direction between the refer- 
ence macroblock and the preceding macroblock. If one 
macroblock_escape is present before the 
macroblock_addressJncrement, the data indicative of 
the actual differentia) in the horizontal direction between 
the reference macroblock and the preceding macrob- 
lock is obtained by adding 33 to the value of the 
macroblock_address_increment 

[0374] The macroblock_quantiser_scale_code is a 
quantization step size set for each macroblock and is 
present only when the macroblock_quant is M." Each 
slice layer has the slice_quantiser_scale_code set 
therefor and indicating the quantization step size there- 
for, and this quantization step size is selected if the 
macroblocK_quantiser_scale_code is set for the. refer- 
ence macroblock. 

[0375] Following the 
macroblock_address_increment, data elements defined 
by the macroblock_modes() function are described. The 
macroblock_modes() function describes data elements 
such as macroblock_type, frame_motion_type, 
field_motion_type, and dct_type as a history stream, as 
shown In Figure 62. 

[0376] The macroblockjype is data indicative of 
the encoding type of a macroblock. The details will be 
discussed later with reference to Figures 65 to 67. 
[0377] If the macrob!ock_motion_forward or the 
macroblock_motion_backward is "1 the picture struc- 
ture is the frame, and the frame_pred_frame_dct is "0," 
-then data elements representing the 
frame_motion_type are described after the data ele- 
ments representing the macroblock_type. The 
frame_pred_frame_dct is a flag indicating whether or 
not the field_motion_type Is present in the bit Stream. 
[0378] The frame_motion_type is a 2-b'rt code indi- 
cating the prediction type of a macroblock In a frame. 
This code is "00" if two prediction vectors are present 
and are of the field-base prediction type, "01" if one pre- 
diction vector Is present and is of the field-base predic- 
tion type, "10" if one prediction vector is present and is 
of the frame-base prediction type, and "11" if one pre- 



diction vector is present and is of a dual prime prediction 
type. 

[0379] If conditions for the description of the 
frame_motion_type are not met, the data elements rep- 

5 resenting the field_motion_Jype are described after the 
data elements representing the macroblock_type. 
[0380] The field_motion_type is a 2-bit code indicat- 
ing a motion prediction for a macroblock In a field. This 
code is "01" if one prediction vector is present and is of 

io the field-base prediction type, "1 0* if two prediction vec- 
tors are present and are of an 18 x 8 macroblock-base 
prediction type, and "11* if one prediction vector is 
present and is of a dual prime prediction type. 
[0381] If the picture structure is the frame, the 

is frame_pred_frame_dct indicates that the 
frame_motlon_type and the dctjtype are present In the 
bit stream, data elements representing the dctjype are 
described after the data elements representing the 
macroblock_type. The dct_type is data indicating 

20 whether the DCT is in a frame or field DCT mode. 

[0382] Referring back to Figure 61 , if the reference 
macroblock is of the forward prediction type or is an 
intramacroblock with a concealing macroblock, data 
elements defined by the motion_vectors(0) are 

25 described. Additionally, if the reference macroblock is of 
the backward prediction type, data elements defined by 
the motion_vectors(1 ) function are described. 
[0383] The motion_vectors(0) function describes 
data elements for a first motion vector, and the 

30 motion_vectors(1) function describes data elements for 
a second motion vector. 

[0384] The motion_vectors(s) function describes 
data elements for motion vectors as shown in Figure 63. 
[0385] If one motion vector is present and the dual 

35 prime prediction mode is not used, data elements 
defined by motion_verticaLfield_select[0][s] and 
motion_vector[0,s] are described. 
[0386] The motion_verticaLfield_select(r][s] is a 
flag indicating whether the first motion vector (that may 

40 be a forward or backward vector) is obtained by refer- 
encing a bottom or a top field. The indicator V indicates 
either the first or second vector, and the "s" indicates 
whether the prediction direction is forward or backward. 
[0387] The motion_yector[r,s] function describes a 

45 data string for motion_code[r][sj[t], a data string for 
• motion_residuai[r][s][t], and data representing dmvec- 

tor[t], as shown in Figure 64. 

[0388] The motion^code[r][s][t] Is data of a variable 
length representing the magnitude of a motion vector 

so between -16 and +16. Thus, the values of the 
motion_code[r][s][t] and the motion_residuaI[r][s][t] can 
describe detailed motion vectors. The dmvectorft] is 
data that operates In the dual prime prediction mode to 
generate motion vectors in one of the fields (for exam- 

55 pie, the top field is referred to as "one of the fields" com- 
pared to the bottom field) by scaling existing motion 
vectors depending on temporal distances while making 
corrections so as to reflect interline vertical offsets 
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between the top field and the bottom field. The indicator 
"r* indicates either the first or second vector, and the "s" 
indicates whether the prediction direction is forward or 
backward. 

[0389] The motion_vector[r,s] function describes a 
data string representing motion_code[r][s]p)] for the hor- 
izontal direction, as a history stream, as shown in Figure 
64. Since the number of bits in both 
motion_residuat[Ol[s][t] and motion_residuaJ[1 ][s][t] is 
indicated by f_code[s][t], the f_code [s][t] of a value 
other than 1 indicates the presence of 
motion_residual[r][s][t] in the bit stream. When 
motion_resldual[r][sJ[0] for a horizontal component Is 
not '1" and the mofion_coder[r][s][0] for a horizontal 
component is not "0/ this means that data elements 
representing the motlon_res!dua![r][s][0] are present In 
the bit stream and the motion vector has a horizontal 
component In this case, data elements representing 
the motion_residual[r][s][0] for a horizontal component 
are described. 

[0390] Following these data elements, a data string 
representing motion_coder[r][s][1 ] for the vertical direc- 
tion is described as a history stream. As described 
above, since the number of bits in both 
motion_residual[0][s][t] and motion_residual[1 )[s][t] is 
indicated by the f__code[s][t], the f_code [s][t] of a value 
other than 1 indicates the presence of 
motion_residual[r][s][t] in the bit stream. When 
motion_residual[r][s][1] Is not "1" and 
motion_code[r][s][1] is not "0," this means that data ele- 
ments representing the motion_residual[r][s][1 ] are 
present in the bit stream and the motion vector has a 
vertical component In this case, data elements repre- 
senting the motion_residual[r][s][1] for a vertical compo- 
nent are described. 

[0391] Next, the macroblock type will be described 
with reference to Figures. 65 to 67. The 
macroblockjype is data of a variable length generated 
by flags such as the macroblock__quant, the 
dct_Jype_flag, the macroblocK_motlon_forward, and the 
macroblock_motion_backward. The macroblock_quant 
is a flag indicating whether or not the 
macroblock_quantiser_scale_code has been set, which 
sets a quantization step size for a macroblock. The 
macroblock_quant is "1" if the 
macroblock_quantiser_scale_code is present in the bit 

-stream. — ~— - - 

[0392] The dct_type_flag -is a -flag indicating 
whether or not the dct_type is present, which indicates 
whether or not the reference macroblock has been 
encoded using the frame or field DCT (in other words, 
this flag indicates whether the reference macroblock 
has been subjected to the DCT). If the dct_type is 
present in the bit stream, the dct_type_flag is "1." The 
macroblock_motion_forward is a flag indicating whether 
or not the reference macroblock has been forward pre- 
dicted. This flag is 1 if the reference macroblock has 
been forward predicted. The 



macroblock_motion_backward is a flag indicating 
whether or not the reference macroblock has been 
backward predicted. This flag is 1 if the reference mac- 
roblock has been backward predicted. 
5 [0393] With the variable- length format, the history 
information can be reduced to diminish the bit rate for 
transmissions. 

[0394] That Is, If the macroblocletype and the 
motion_vectors() are transferred and not the 

io quantiser_scaJe_code, the bit rate can be lessened by 
setting the slice_quantiser_scaJe_code at "00000." 
[0395] In addition, if only the macroblock_type is 
transferred and not the motion_vectorsO, the 
quantiser_scale_code, and the dcMype, the bit rate 

is can be diminished by using "not coded" as the 
macroblock_type. 

[0396] Further, if only the picture_coding_type is 
transferred and not the slice() and subsequent informa- 
tion, the bit rate can be reduced by using the 

20 picture_data() that does not have the slice start code. 
[0397] In the above description, to prevent continu- 
ous 23 bits of Os from appearing in the user_data, "1" is 
inserted into the data at intervals of 22 bits, but the inter- 
val may not be 22 bits. In addition, instead of counting 

25 the number of continuous Os, Byte_allign can be exam- 
ined for insertion. 

[0398] Further, although the MPEG prohibits the 
occurrence of continuous 23 bits of Os, it actually deals 
only with continuous 23 bits of Os starting at the head of 
30 the byte and not with such bits starting at the middle of 
the byte. Thus, "1 " may be inserted at positions other 
than LSBs, for example, at intervals of 24 bits. 
[0399] Additionally, in the above description, the 
history information Is in a form similar to video elemen- 
ts tary streams, but it may be substantially in the form of 
packetized elementary streams or transport streams. In 
addition, although the user_data In the elementary 
stream precedes the picture_data, it may be located 
somewhere else. ' 
40 [0400] The transcoder 101 described in Figure 15 
provides the subsequent process with encoding param- 
eters for a plurality of generations as history informa- 
tion, but all the above described history information is 
not required. For example, if this transcoder is followed 
45 by a recording and reproducing system including a 
large-capacity recording medium that is relatively free 
from limitations on the storage capacity, no problem 
occurs if all the above described information is 
described in the encoding parameters. If, however, the 
so transcoder is followed by a recording and reproducing 
system including a recording medium of a relatively 
small capacity, only required history information is desir- 
ably described in the encoding parameters instead of all 
the history information, in order to more or less reduce 
55 the data rate of encoded streams. By way of another 
example, if this transcoder is followed by a transmission 
path including a large-transmission-capacity recording 
medium that is relatively free from limitations on the 



35 



69 



EP 1 069 779 A1 



70 



transmission capacity, no problem occurs if ail the 
above described information is described in the encod- 
ing parameters. If, however, the transcoder is followed 
by a transmission path having a relatively small trans- 
mission capacity, only required history information is 
desirably described in the encoding parameters instead 
of all the history Information, in order to more or less 
reduce the data rate of encoded streams. 
[0401 ] The present invention is characterized in that 
history information required for each of various applica- 
tions following the transcoding device is described in the 
encoded stream adaptively and selectively depending 
on the application. To achieve this, this embodiment 
describes the information re-coding_streamJnfo in the 
encoded stream. 

[0402] A syntax and data elements for the re- 
codingLStrearnJnfo will be described below in detail 
with reference to Figure 68. 

[04O3] As shown in Figure 68, the 
re_coding_stream_jnfo() function is composed of the 
user_data_start_code, the re__codlng_streamJnfo_ID, 
the red_bw_flag, the red_bw_in dicato r, the marker__bit, 
the num_other_brts, the num_mv_bits, the 
num_coefJ>lts, etc. 

[0404] The user_data_start_code is a start code 
indicative of the start of the user_data. The 
re_coding_stream_jnfoJD is a 16-bit integer used to 
identify the re_coding_stream_info() function. Specifi- 
cally, this integer has a value of "1 001 0001 1 1 1 0 1 1 00" 
(0x91 ec). 

[0405] The red_bw_flag is a 1 -bit flag that is 0 if the 
coding parameters for all the history information are 
transmitted and that is 1 if the coding parameters for the 
history information are selectively transmitted. Specifi- 
cally, as shown in Figure 69, if the red_bw_flag is 1, 
checking the red_bw_indicator following this flag ena- 
bles determination of which of five data sets is used to 
transmit the corresponding coding parameters for the 
history information. This data set contains information 
for determining a combination of coding parameters to 
be transmitted with the re-coded encoded stream. 
Accordingly, the coding parameters to be described in 
the encoded stream are selected in accordance with 
this data set 

[0406] The redjbwjndicator is a 2-bit integer act- 
ing as an indicator for defining a data set for the coding 
- parameters. Specifically, the red^bw_indicator is data 
indicating which of data sets 2 to 5 is represented as 
shown in Figure 69. 

[0407] Thus, by referencing the red_bw_flag and 
the red_bw_lndicator, which are described In the 
encoded stream, it can be determined which of the five 
data sets Is used to transmit the coding parameters for 
the history information. 

[0408] Next, the coding parameters for the history 
Information transmitted in each data set will be 
explained with reference to Figure 70. 
[0409] The history information can be roughly clas- 



sified into information in pictures and information in 
macroblocks. Information in slices is obtained by col- 
lecting macro block information contained in such infor- 
mation, and information in GOPs is obtained by 
5 collecting the information in pictures contained in such 
information. 

[0410] Since the information in pictures is transmit- 
ted only once per frame, its bit rate is not so high com- 
pared to the history information inserted into the 

io encoded stream. On the contrary, since the information 
in macroblocks is transmitted for each macroblock, if, for 
example, in a video system with 525 scan lines for one 
frame and with a field rate of 60 fields/sec, 720 x 480 
pixels are contained in one frame, then the information 

is in macroblocks must be transmitted 1 ,350 (=(720/1 6) x 
(480/16)) times per frame. As a result, a relatively large 
part of the history information is occupied by the infor- 
mation in macroblocks. 

[0411] Accordingly, in this embodiment, as the his- 

20 tory information inserted into the encoded stream, at 
least the information in pictures is constantly transmit- 
ted but the information in macroblocks is selectively 
transmitted depending on the application, thereby 
reducing transmitted information. 

25 [0412] As shown in Figure 70, the information in 
macroblocks transmitted as the history information 
includes, for example, the num_coef_bits, the 
num_mv_bits, the num_other_bits, the q_scale_code, 
the q_scale_type, the motion_type, the 

30 mv_vert_field_sel[][], the mvQQQ, the mb_mfwd, the 
mb_mbwd, the mb_pattem, the code d_b lock_patte rn , 
the mbjntra, sllce_start, the dct_type, the mb_quant, 
the skipped_rnb, etc. These information is represented 
by using the element of macrobiock_rate_information 

35 defined in SMPTE-327M. 

[0413] The num_coef_brts represents the amount 
of macroblock codes required for DCT coefficients. The 
num_mv_bits represents the amount of macroblock 
codes required for motion vectors. The num_other__bits 

40 represents the amount of macroblock codes other than 
the num_coef_bits and the num_mv_bits. 
[0414] The q_scale_code represents q_sca!e_code 
applied to the macroblock. The motionjype represents 
the type of the motion vectors applied to the macrob- 

45 lock. The mv_vert_field_selQn represents a field selec- 
tion for the motion vectors applied to the macroblock. 

- [0415] The-mvODD represents the motion vectors 
applied to the macroblock. The mb_mfwd Is a flag Indi- 
cating that the prediction mode for the macroblock is for- 

so ward prediction. The mb__mbwd is a flag indicating that 
the prediction mode for the macroblock Is backward pre- 
diction. The mb_pattern is a flag indicating the presence 
a non-zero DC coefficient of the macroblock. 
[0416] The coded_block_pattern is a flag indicating 

55 the presence of a non-zero DC coefficient of the mac- 
roblock for each DCT block. The mbjntra is a flag Indi- 
cating whether or not the macroblock is an intra_macro. 
The slice_start is a flag indicating whether or not the 
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macroblock is the head of a slice. The dct_type is a flag 
indicating whether or not the macroblock is the field_dct 
or the flame_dct 

[0417] The mb_quant is a flag indicating whether or 
not the macroblock transmits the quantiser_scale_code. 
The skipped_mb is a flag indicating whether or not the 
macroblock is skipped macroblock. 
[0418] Not all these coding parameters are con- 
stantly required, but required coding parameters vary 
depending on the application following the transcoder. 
For example, coding parameters such as the 
num_coef_bits and the slice_start are required for appli- 
cations that Involves a transparent request that requests 
a bit stream to be recovered to its original form as 
closely as possible during re-coding. The "transparent 
request" enables an output bit stream to be generated 
with its image quality prevented from degradation com- 
pared to an input bit stream. 

[0419] That is, applications that request only a 
change in bit rate for a transcoding process do not 
require the coding parameters such as the 
num_coet_brts and the slice_start In addition, if there 
are very strict limitations on the transmission path, cer- 
tain applications require only the encoding type of each 
picture. 

[0420] In view of these circumstances, this embodi- 
ment provides a data set such as that shown in Figure 
70, as an example of a data set for coding parameters 
that is used in transmitting history information. 
[0421] In Figure 70, the value "2" corresponding to 
coding parameters in each data set means that the cor- 
responding Information is present in the encoded 
stream and available as the history information, and "0" 
means that the corresponding information is absent 
from the encoded stream. "1" means that the corre- 
sponding information is present for support for the pres- 
ence of other Information or for the syntax, but has no 
meaning, for example, no relations with the source bit 
stream information. For example, the slice_start is "1" 
for the macroblock at the head of a slice for transmitting 
history information but has no meaning as history infor- 
mation if the slice does not necessarily maintain the 
same locational relationship with the original bit stream. 
[0422] In this embodiment, the coding parameters 
(num_coef_bits, num_mv_bits, num_other_bits), 
(q_scale_code, q_scale_type) p (motion_type, 
~ mv_vert_fieldlselOD; rnvDDD), (mbimfwd, mb_mbwd), 
(mb_pattern), (coded_block .pattern), (mbjntra), 
(slice_start), (dct_type), (mb_quant), and (skipped_mb) 
are selectively described in the encoded stream 
depending on the selected data set. 
[0423] The data set 1 is intended to reconstruct a 
completely transparent bit stream. The data set 1 ena- 
bles accurate transcoding that outputs a bit stream with 
little image quality degradation compared to an input bit 
stream. The data set 2 is also Intended to reconstruct a 
completely transparent bit stream. The data set 3 can- 
not reconstruct a completely transparent bit stream but 



reconstructs a substantially visually transparent bit 
stream. The data set 4 is inferior to the data set 3 in 
terms of transparency but enables reconstruction of a 
bit stream with no visual problem. The data set 5 is infe- 

5 rior to the data set 4 in terms of transparency but can 
reconstruct a bit stream with a little history information 
despite the incompleteness of the reconstruction. 
[0424] These data sets are functionally higher as 
their data set numbers are smaller, but functionally 

io higher data sets require a larger capacity for transmis- 
sion of history information. Consequently, the transmit- 
ted data set is determined taking into consideration the 
assumed application and the capacity available for the 
history information. 

15 [0425] Of the five data sets shown in Figure 70, for 
the data set 1 , the red_bw_flag is 0, while for the data 
sets 2 to 5, the red_bw_flag is 1. In contrast, the 
red_bw_indicator is 0 for the data set 2, 1 for the data 
set 3, 2 for the data set 4, and 3 for the data set 5. 

20 [0426] Thus, the red_bw_indicator is specified if the 
redj>w_flag is 1 (that Is. for the data sets 2 to 5). 
[0427] Furthermore, if the ref_bw_f lag is 0 (the data 
set 1), the marker_bit, the num_other_b*rts, the 
num_mv_bits, and the num_coef_bits are described for 

25 each macroblock. These four data elements are not 
described in the encoded stream for the data sets 2 to 5 
(that is, if the red_bw_flag is 1 ). 

[0428] For the data set 5, the other syntax elements 
are not transmitted Including the picture_dataO function 

30 (see Figure 59). That is, no encoding parameters are 
transmitted for a plurality of slice () functions contained 
in the p!cture_dataO function. Accordingly, for the data 
set 5, selected history information is intended to trans- 
mit only coding parameters in pictures such as the 

35 picture_type. 

[0429] For the data sets 1 to 4, coding parameters 
are present for the plurality of slice() functions contained 
in the picture_data() function. However, address infor- 
mation for slices determined by the slice() function and 

40 for slices in the source bit stream depend on the 
selected data set For the data set 1 or 2, the address 
information for the slices in the source bit stream for the 
history information must be identical to the address 
information for the slices determined by the sfice() func- 

45 tion. 

[0430] The syntax elements of the macroblock() 
fu nction (see Figu re 6 1 ) "depend On the selected data 
set. The mactoblocK_escape, 

macroblock_address_increment, and 

so macroblock_modes() functions are constantly present 
In the encoded stream. The effectiveness of the 
m aero block_e scape and 
macroblock_addressJncrement as information, how- 
ever, is determined by the selected data set. If the data 

55 set 1 or 2 is used for the coding parameters for the his- 
tory information, information identical to the 
skipped_mb in the source bit stream must be transmit- 
ted. 
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[0431] For the data set 4, the motion_vectorsO 
function is absent from the encoded stream. For the 
data sets 1 to 3, the rnacroblockjype of the 
macroblocKjTiodesO function determines the presence 
of the motionj/ectorsO function in the encoded stream. 
For the data set 3 or 4, the coded J)lock _pattern() func- 
tion is absent from the encoded stream. For the data 
sets 1 and 2, the rnacroblockjype of the 
macroblocK_modesO function determines the presence 
of the coded_b!ock __pattern() function in the encoded 
stream. 

[0432] The syntax elements of the 
macroblock_modes0 function (see Figure 62) depend 
on the selected data set. The rnacroblockjype exists 
constantly. For the data set 4, the fiame_motion Jype, 
the field_motlon Jype, and the dct_type are absent from 
the encoded stream. 

[0433] The effectiveness as information of a param- 
eter obtained from the rnacroblockjype is determined 
by the selected data set. 

[0434] For the data set 1 or 2, the macroblock quant 
must be the same as in the source bit stream. For the 
data set 3 or 4, the macroblock_quant represents the 
presence of the quantiser__scale_code in the macrob- 
lock() function and need not be the same as in the 
source bit stream. 

[0435] For the data sets 1 to 3, the 
macroblock_motion Jorward must be the same as in the 
source bit stream. This is not necessary for the data set 
4 or 5. 

[0436] For the data set 1 or 2, the 
macroblock_pattern must be the same as In the source 
bit stream. For the data set 3, the macroblock_pattern is 
used to indicate the presence of the dct_type. The rela- 
tions for the data sets 1 to 3 are not established for the 
data set 4. 

[0437] If the data sets 1 to 3 are used for the coding 
parameters for the history information, the 
macroblock_intra must be the same as in the source bit 
stream. This does not apply to the data set 4. 
[0438] Next, processing executed by the transcoder 
101 on an encoded stream containing information on 
the data sets and to generate an encoded stream based 
on the set data sets will be described with reference to 
the transcoder 101 shown in Figure 15. 
[0439] The example of a transcoder shown in Fig- 
— ure 15 receives an encoded stream ST (3rd) generated 
by the third-generation encoding process to convert the 
GOP structure or/and bit rate thereof in order to gener- 
ate a new encoded stream ST (4th). 
[0440] First, the decoding device 102 extracts from 
the third-generation encoded stream St (3rd) third-gen- 
eration coding parameters used to encode this encoded 
stream St (3rd), and decodes the encoded stream ST 
(3rd) based on the extracted coding parameters to gen- 
erate a baseband video signal. Further, the decoding 
device 102 outputs the extracted third-generation cod- 
ing parameters (3th) to the history information -multi- 



plexing device 103. Further, the decoding device 102 
extracts the user_data() from the picture layer of the 
third-generation encoded stream ST (3rd) to supply the 
data to the history decoding device 1 04. 
5 [0441] The history decoding device 104 extracts the 
history_streamO from the user_data() supplied by the 
decoding device 102. Since the history_streamO is a 
stream consisting of variable-length-encoded data ele- 
ments, the history decoding device 104 carries out a 
10 variable-length decoding process on the 
history_stream()- As a result, a stream can be gener- 
ated which consists of data elements each having a pre- 
determined data length. Next, the history decoding 
device 104 parses a syntax for the variable-length 
15 decoded data stream. The parsing refers to interpreta- 
tion of the syntax for the stream. 
[0442] During the parsing process, the history 
decoding device 104 references the red_bw_flag and 
red_bw_indicator described in the 

20 re_codi ng_stream_info () in the history_stream() func- 
tion. By referring the red_bw_flag and red_bwJndicator 
extracted from the stream, the history decoding device 
104 determines which of the five data sets are set for 
the received history_stream(). Thus, by determining the 
25 type of the data set from the red_bw_flag and the 
red_bw_jndicator, the history decoding device 104 can 
determine which coding parameters are contained in 
the history_stream(). 

[0443] Specifically, if the red_bw_flag = 0, the data 
30 set 1 is set Thus, all the coding parameters are 
described in the history_stream() as the picture_data() 
function, including the num_coef_bits, the 
num_mvjDits, the num__other_b"tts, the q_scale_code, 
the q_scale_type, the motion_type, the 
35 mv_vertJield_selOQ, the mvflQD. the mb_mfwd, the 
mb_mbwd, mb_pattern, the coded_bIock__pattem, 
mbjntra. the sllce_start, the dctjype, the mb_quant, 
and the skipped_mb. 

[0444] If the red_bw_flag = 1 and the 
40 red J>wJndicator = 0, the data set 2 is set. Accordingly, 
coding parameters are described in the 
history_stream() as the picture_data() function, includ- 
ing the q_scale_code, the q__scale_type, the 
motionjype, the mv_vert_fieloLseIQO, the mvO|]0, the 
45 mb_mfwd, the mb_mbwd, the mb_pattern, the 
coded J>lock_pattem, the mbjntra, the slice_start, the 

dctjype, the mb~quant, and the skipped_mb.- 

[0445] . If the red_bw_flag. = 1 and the 
red_bw_indicator = 1 , the data set 3 is set. Accordingly, 
so coding parameters are described in the 
history_streamO as the picture jiata() function, Includ- 
ing the q_scale_code, the q_scale_type, the 
motlon_type, the mv_yert_fie1d_selQ[], the rnvODQ, the 
mb_mfwd, the mb_mbwd, the mb_pattern, the 
55 mbjntra, the slice_start, the dctjype, the mb_quant, 
and the skipped_mb. 

[0446] If the red_bwjlag = 1 and the 
red_bwJndicator = 2, the data set 4 is set. Accordingly, 
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coding parameters including the q_scale_code and the 
q_scale_type are described in the history__stream() as 
the picture_data() function. 

[0447] If the red_bw_flag = 1 and the 
redjwjndicator = 3, the data set 5 is set Accordingly, 
no coding parameter is described in the 
history_streamO as the picture_data() function. 
[0448] The history decoding device 1 04 references 
the information in the red_bw_flag and 
red_bwJndicator to extract the coding parameters from 
the history_stream(). In the embodiment of the trans- 
coder shown in Figure 15, the input encoded stream 
supplied to the history decoding device 104 transcoder 
has been generated by the third-generation encoding 
process, so that output history information is the first- 
and second-generation coding parameters. 
[0449] The history information-multiplexing device 
103 multiplexes the third-generation coding parameters 
(3th) supplied by the decoding device 102 and the past 
coding parameters (1st, 2nd) supplied by the history 
decoding device 1 04, into the baseband video data sup- 
plied by the decoding device 102, in accordance with 
such a format as shown in Figures 68 to 73. 
[0450] The history information-separating device 

105 receives the baseband video data supplied by the 
history information-multiplexing device 1 03 and extracts 
the first-, second-, and third-generation coding parame- 
ters (1st, 2nd, 3rd) from the baseband video data to 
supply them to the encoding device 1 06. 

[0451] The encoding device 1 06 receives the base- 
band video data and the coding parameters (1st, 2nd, 
3rd) from the history Information-separating device 105 
to re-code the baseband video data based on the 
received coding parameters. In this case, the encoding 
device 106 selects coding parameters optimal for the 
encoding process, from the coding parameters (1st, 
2nd, 3rd) generated during the past encoding proc- 
esses and the coding parameters newly generated from 
the supplied baseband video data. The encoding device 

106 carries out the above described "normal encoding 
process" if it uses for the encoding the coding parame- 
ters newly generated from the supplied baseband video 
data. The encoding device 106 carries out the above 
described "parameter-reused encoding process" if it 
uses any of the past coding parameters (1st, 2nd, 3rd). 
[0452] The computer 100 provided on the network 

-controls the decoding process executed by the decod- 
ing device 102 and the encoding process executed by 
the encoding device 106. For example, the computer 
1 00 detects the capacity of the transmission path for 
transmitting an encoded stream output by the encoding 
device 1 06 and selects an appropriate one of the five 
data sets depending on the transmission capacity. It 
also detects the storage capacity of a device connected 
to an output of the encoding device 106 and selects an 
appropriate one of the five data sets depending on the 
storage capacity. 

[0453] The encoding device 106 receives informa- 



tion indicative of the data set from the computer 100 to 
generate the red_bw_flg and the red_bw_jndicator 
based on this information. If the information provided by 
the computer 100 Indicates the data set 1, the 

5 red_bw_flag = 0. If the information indicates the data set 
2, the red_bw_flag = 1 and the red_bw_indicator = 0. If 
the information Indicates the data set 3, the red_bw_flag 
= 1 and the redjowjndicator = 1. If the information indi- 
cates the data set 4, the red_bw_flag = 1 and the 

7 0 red_J>w_indicator = 2. If the information indicates the 
data set 5, the red_bwjlag = 1 and the 
red Jawjndlcator = 3. 

[0454] Depending on the determined values of the 
red_bw_fiag and the red_bw_indicator f the encoding 

15 device 1 06 selects coding parameters for description in 
the encoded stream as the htstory_stream(), and 
describes the selected coding parameters in the 
encoded stream as the history_stream(), while describ- 
ing the red_bw_flag and the red_bw_indlcator in the 

20 encoded stream as the re coding _stre am infoQ. The 
coding parameters transmitted as the hlstory_stream() 
are selected for each of the first, second, and third gen- 
erations of coding parameters. 

[0455] If the red_bw_flag = 0, then, the encoding 

25 device 106 describes ail the coding parameters in the 
encoded stream as the picture_data() function, includ- 
ing the num_coef_bits, the num_mv_bits, the 
num_other_bits, the q_scale_code, the q_scale_type, 
the motionjype, the mv__yert_field_selOQ, the mvQDD, 

30 the mb_mfwd, the mb_mbwd, mb_pattern, the 
coded_block_pattern, mbjntra, the slice_start, the 
dctjype, the mb_quant, and the sklpped_mb. 
[0456] If the red_bw_flag = 1 and the 
red_bw_indicator = 0, the encoding device 106 

35 describes coding parameter in the history_stream() as 
the picture_dataQ function, including the q_scale_code, 
the q_scalejype, the motionjype, the 
mv_vert_field_sel[|n» the mvQQQ, the mb_mfwd, the 
mb_mbwd, the mb_pattern, the coded_block_pattern, 

40 the mbjntra, the slice_start, the dct_type, the 
mb_quant, and the skipped_mb. 

[0457] If the reoLbwJIag = 1 and the 
red_bwjndicator = 1, the encoding device 106 
describes coding parameters in the history__stream() as 

45 the picture_data() function, including the q__sca!e_code, 
the q_scale_type, the motionjype, the 
mv_vertTield_sel[|0. the mv|]Q0, the mb_mfwd, the 
mb_mbwd, the mb_pattem, the mbjntra, the 
slice_start, the dctjype, the mb_quant, and the 

50 skipped_mb. 

[0458] W the red_bwjlag = 1 and the 
red_bwJndicator = 2, the encoding device 106 
describes coding parameters including the 
q_scale_code and the q_scalejype, in the 

55 history_stream() as the picture_data() function. 

[0459] If the red_bwjlag = 1 and the 
red_bwJndicator = 3, the encoder 106 describes no 
coding parameter in the history_stream() as the 
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picture_data() function. 

[0460] That is, the encoding device 106 selects 
from the coding parameters transmitted as the 
history_stream() based on the information indicating 
data set specified by the computer 1 00, instead of trans- 
; mitting all the supplied past coding parameters as the 
history__stream(). Thus, based on the received informa- 
tion Indicating the data set, the encoding device 1 05 can 
generate an encoded stream containing 
hlstory_stream() with various data capacities. In addi- 
tion, the encoding device 105 can generate an encoded 
stream containing history jstreamO consisting of an 
amount of data appropriate for the transmission capac- 
ity of a transmission media following the encoding 
device 105, for the storage capacity of a recording 
media, or tor the application. According to the trans- 
coder of this embodiment, the coding parameters trans- 
mitted as the history_stream() in this way are selected 
in accordance with an application connectivety following 
the encoding device. Therefore, the history correspond- 
ing to the application can be transmitted in an optimal 
data amount 

[0461] Next, a format in which the history informa- 
tion Is multiplexed into the baseband video signal output 
by the history information-multiplexing device 103 will 
be described with reference to Figu res 71 to 74. 
[0462] Figure 71 is a diagram showing the format 
"Re_Coding information BUS macroblock format,' in 
which the history information is multiplexed into the 
baseband video signal for transmission. This mask 
block is configured by 1 6 x 16 (=256) bits. In Figure 71 , 
the 32 bits shown in each of the third and fourth rows 
from the top are picrate_element. Picture rate elements, 
shown in Figure 72 to 74, are described in the 
picrate_element The 1-bit red_bw_flag is specified in 
the second row from the top in Figure 72, and the 3-bit 
red_bwJndicator is specified in the third row from the 
top. That is, the flags red_bw_flag and red_bwJndicator 
are transmitted as the picrate_element in Figure 71 . 
[0463] The other data in Figure 71 will be explained 
below. SRIB__sync_code is a code representing that the 
first row of a macroblock in this format is aligned at the 
left end of the stream. Specifically, this code is set at 
•1 1 1 1 1 ." If the picture_structure indicates the frame pic- 
ture structure (that is, its value is "1 1"), fr_fl_SRIB is set 
to 1, indicating that more than 16 lines of Re_Coding 
-Information Bus macroblock are transmitted. If the 
picture_structure doe not indicate the frame structure, 
the fr_fl_SRIB is set to 0, indicating that more than 16 
lines of Re_Coding Information Bus are transmitted. 
This mechanism allows the Re_Coding Information Bus 
to be locked to corresponding pixels in a spatially and 
temporally decoded video frame or field. 
[0464] SRIB_top_fteld_first is set at the same value 
as top_field_first held in the source bit stream and rep- 
resents, with repeat_first_field, the temporal alignment 
of the Re_Coding Information Bus for a related video. 
SRIB_repeat_field_first is set at the same value as 



repeat_first_field held in the source bit stream. The con- 
tents of the Re_Coding Information Bus for a first field 
must be repeated as indicated by this flag. 
[0465] 422_420_chroma represents whether the 

s source bit stream is of 4:2:2 or 4:2:0. The 
422_420_chroma of 0 represents that the bit stream is 
of 4:2:0 and that a chromatic aberration signal has been 
up-sampled so as to output a 4:2:2 video. The 
422__420_chroma of 0 represents that the chromatic 

w aberration signal has not been filtered. 

[0466] rolling_SRIB_mb_ref represents a 16-bit 
modulo 65521, the value of which is incremented con- 
sistently with the number of macroblocks. This value 
must be continuous across frames of the frame picture 

is structure. Otherwise, it must be continuous across 
fields. It is initialized to a predetermined value between 
0 and 65520. This allows a unique identifier of the 
Re_Coding Information Bus to be incorporated in a 
recorder system. 

20 [0467] The meaning of the other data for the 
Re_Coding Information Bus macroblock Is as described 
above and is thus omitted. 

[0468] As shown in Figure 75, the 256-bit data for 
the Re_Coding Information Bus in Figure 71 is 

25 arranged, by one bit at a time, in Cb[0][0], 
Cr[0][0],Cb[1][0], and Cr[1l[0], which are LSBs of chro- 
matic aberration data. Since the format shown in Figure 
75 allows 4-bit data to be sent, the 256-bit data in Figure 
71 can be transmitted by sending the format in Figure 

30 75 sixty-four times (=256/4). 

[0469] According to the transcoder of the present 
invention, the coding parameters generated during the 
past encoding processes are reused for the current 
encoding process, thereby precluding image quality 

35 degradation despite repetition of the decoding and 
encoding processes. That is, the present invention can 
restrain the accumulation of image quality degradation 
originating from repetition of the decoding and encoding 
processes. 

40 [0470] Figure 76 and 77 show examples of configu- 
rations of a video tape recorder with the transcoder of 
the present invention applied thereto. Figure 76 shows 
an example of a configuration of a recording system of 
a video tape recorder 601 . Figure 77 shows an example 

45 of a configuration of a reproduction system of the video 
tape recorder 601 . 

[0471] The video tape recorder-601 in Figure 76 is- 
configured by a transcoder 101R,a channel encoding 
device 602, and a recording head 603. The transcoder 

so 1 01 R is configured basically in the same manner as the 
transcoder shown in Figure 37. In this example of con- 
figuration, the transcoder 1 01 R converts a bit stream ST 
of a Long GOP into one of a Short GOP. 
[0472] A fourth-generation encoded stream ST out- 

55 put from the encoding device 106 of the transcoder 
101R is supplied to the channel encoding device 602. 
As described above, the user_data containing the first- 
to third-generation coding parameters is recorded in the 
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user data area of the picture layer of the fourth-genera- 
tion encoded stream ST. 

[0473] The channel encoding device 602 appends 
a parity code for error correction to the input fourth-gen- 
eration encoded stream subsequently carries out chan- 5 
nel encoding using, for example, the NRZI modulation 
method, and then supplies the encoded stream to a 
recording head 603. The recording head 603 records 
the input encoded stream on a magnetic tape 604. 
[0474] As shown in Figure 77, in the reproduction 1 o 
system, a reproduction head 611 generates a signal 
from the magnetic tape 604 to supply it to a channel 
decoding device 612. The channel decoding device 612 
channel-decodes the signal supplied by the reproduc- 
tion head 61 1 and then corrects errors therein using the is 
parity. 

[0475] The fourth-generation encoded stream ST 
output by the channel decoding device 612 is input to a 
transcoder 1 01 R The transcoder 1 01 P has a basic con- 
figuration similar to that of the transcoder shown in Fig- 20 
ure 37. 

[0476] The decoding device 102 of the transcoder 
101P extracts from the fourth-generation encoded 
stream the user data user_data containing the first- to 
third-generation coding parameters, and supplies the 25 
data to the history encoding device 104 and the encod- 
ing device 106. The history decoding device 104 
decodes the input user data user_data to deliver the 
obtained first- to third-generation coding parameters to 
the encoding device 1 06. 30 
[0477] The decoding device 102 also decodes the 
fourth-generation encoded stream ST to output the 
baseband video signal and the fourth-generation coding 
parameters. The baseband video signal is supplied to 
the encoding device 106, while the fourth-generation 35 
coding parameters are delivered to the encoding device 
106 and the history encoding device 107. 
[0478] The history encoding device 107 converts 
the input fourth-generation coding parameters into the 
user data user_data to supply the data to the encoding 40 
device 1 06. 

[0479] As described above, the controller 70 for the 
encoding device 106 determines whether or not the pic- 
ture type of each picture determined from the GOP 
structure specified by the operator is the same as the 45 
picture type contained in the history information (user 
-data user-data). Depending on a result of the determi- 
nation, the encoding device 106 carries out the above 
described "normal encoding process" or "parameter- 
reused encoding process." After this process, the so 
encoding device 106 outputs the fourth-generation 
encoded stream ST with the Long GOP into which the 
Short GOP has been converted. The user data 
user_data in the encoded stream ST has the first- to 
fourth-generation coding parameters recorded therein 55 
as history information. 

[0480] Although the video tape recorder 601 shown 
in Figures 76 and 77 records the history information in 



the user_data of the picture layer, the history informa- 
tion can be recorded in an area of the magnetic tape 
604 which is different from an area for video data. Fig- 
ures 78 and 79 show examples of configurations of the 
video tape recorder 601 in this case. Figure 78 shows 
an example of a configuration of a recording system of 
the video tape recorder 601 . Figure 79 shows an exam- 
ple of a configuration of a reproduction system of the 
video tape recorder 601. 

[0481] As shown in Figure 78, in the video tape 
recorder 601 , user data user_data output by the decod- 
ing device 1 02 of the transcoder 1 01 R is input to the his- 
tory decoding device 104, which then decodes past 
coding parameters (in this example, the first- and sec- 
ond-generation coding parameters) for supply to the 
encoding device 106. In addition, this example need not 
record history information on the magnetic tape 604 as 
user data user_data and thus employs only the history 
VLC 21 1 instead of the entire history encoding device 
107 shown in Figure 15. The history VLC 211 is sup- 
plied with coding parameters (in this case, third-genera- 
tion coding parameters) output by the decoding device 
1 02 and with the coding parameters (in this case, the 
first- and second-generation coding parameters) output 
after being decoded from the user data userjJata by 
the history decoding device 104. The history VLC 211 
variable-length encodes the first- to third-generation 
coding parameters to generate the history_stream 
shown in Figures 40 to 46 or 47 for supply to a multi- 
plexer 621 . 

[0482] The multiplexer 621 also receives a fourth- 
generation encoded stream ST output by the encoding 
device 106. The multiplexer 621 multiplexes the 
encoded stream (bit stream) supplied by the encoding 
device 106 into an area that is safer than an area in 
which the history supplied by the history VLC 211 is 
recorded. 

[0483] For example, as shown in Figure B0, the 
video stream output by the encoding device 106 is 
recorded on the magnetic tape 604 near a sync code, 
while the history_stream output by the history VLC 21 1 
is recorded more remotely from the sync code than the 
video stream. When the video stream is retrieved dur- 
ing, for example, special reproduction, the sync code is 
first detected and the subsequent video stream is 
retrieved using the sync code as a reference. Conse- 
quently, by locating the video stream closer to the sync - 
code, the vide data can be reliably reproduced even 
during fast reproduction. The history_stream is not 
required for fast reproduction. Thus, adverse effects are 
avoided even when the history_stream is located away 
from the sync code. 

[0484] The signal multiplied by the multiplexer 621 
is input to the channel encoding device 602. After chan- 
nel encoding, the recording head 603 records the signal 
on the magnetic tape 604. 

[0485] In this manner, this example multiplexes the 
history_stream in the position different from that for the 



41 



81 



EP 1 069 779 A1 



82 



video data. Consequently, even if the start code 
appears at that position, it can be sufficiently distin- 
guished from the video data. As a result, this example 
does not require Insertion of the. marker bit as required 
for conversion of the history_stream into the 
converted_history_stream. 

[0486] In addition, the coding parameters can be 
supplied to the multiplexer 621 for multiplexing instead 
of being converted into the format of the history_strearn. 
In this case, however, the data Is not compressed, 
thereby increasing the amount of data for the coding 
parameters to reduce the operational efficiency of the 
magnetic tape 604. Thus, the history VLC 21 1 Is desir- 
ably used to compress the data into the format of the 
history_stream before multiplexing. 
[0487] As shown in Figure 79, in the reproduction 
system of the video tape recorder 601, the signal repro- 
duced from the magnetic tape 604 by the recording 
head 611 is channel-decoded by the channel decoding 
device 612. A demultiplexer 631 is channel-decoded by 
the channel decoding device 612. The demultiplexer 
631 separates the fourth-generation encoded stream 
ST supplied by the channel decoding device 612, into 
the video stream and the history_stream, and supplies 
the video stream to the decoding device 102, while 
delivering the history_stream to the history VLD 203. 
[0488] That is, this example employs only the his- 
tory VLD 203 instead of the entire history decoding 
device 104 shown in Figure 15. 

[0489] The history VLD 203 variable-length 
decodes the history_stream to output the obtained first- 
to third-generation coding parameters to the encoding 
device 106. 

[0490] In addition, the history_stream output by the 
demultiplexer 631 is input to a converter 21 2\ The con- 
verter 212' and a following user data formatter 213' are 
separate from the converter 212 and user data format- 
ter 213 (see Figure 15) built into the history encoding 
device 107 but provide the same functions. 
[0491] That is, the converter 212' adds the marker 
bit to the history stream input by the demultiplexer 631 
to generate the cohverted_history_stream and outputs 
it to the user data formatter 213*. The user data format- 
ter 213' converts the input converted_history_stream 
into the user_data to output the data to encoding device 

106. The user_data contains the first- to third-genera- 
tion coding parameters. — - 

[0492] The decoding device 102 decodes the video 
stream input by the demultiplexer 631 to output the 
baseband video signal to the encoding device 1 06. The 
decoding device 102 also supplies the fourth-genera- 
tion coding parameters to the encoding device 106, 
while outputting them to the history encoding device 

107. The history encoding device 107 generates the 
user_data from the input fourth-generation coding 
parameters to output the data to the encoding device 
106. 

[0493] Similarly to the encoding device 1 06 in Fig- 



ure 77, the encoding device 106 executes the •normal 
encoding process* or the ■parameter-reused encoding 
process" to output a fifth-generation encoded stream 
ST. This fifth-generation encoded stream ST has the 
5 first- to fourth-generation coding parameters recorded 
in the user_data of the picture layer thereof. 
[0494] As appreciated from the above description, 
according to the transcoder of the present Invention, the 
coding parameters generated during the past encoding 
io processes are described in the user data area of the 
encoded stream generated during the current encoding 
process, and the generated bit stream is an encoded 
stream conformable to the MPEG standard. Conse- 
quently, any current decoder can be used for the decod- 
es ing process. Further, the transcoder according to the 
present invention requires no dedicated line or the like 
for transmitting the coding parameters for the past 
encoding processes, whereby conventional data stream 
transmission environments can be directly used to 
20 transmit the past coding parameters. 

[0495] According to the transcoder of this embodi- 
ment, the coding parameters generated during the past 
encoding processes are selectively described in the 
encoded stream generated during the current encoding 
25 process. As a result, the past coding parameters can be 
transmitted without the need to extremely increase the 
bit rate of the output bit stream. 

[0496] According to the transcoder of this embodi- 
ment, coding parameters optimal for the current encod- 
30 ing process are selected from the past and current 
coding parameters for encoding. Consequently, the 
accumulation of image quality is avoided despite repeti- 
tion of the decoding and encoding processes. 
[0497] According to the transcoder of this embodi- 
es ment, coding parameters optimal for the current encod- 
ing process are selected from among the past coding 
parameters in accordance with the picture type for 
encoding. Consequently, levels of deterioration in image 
quality are prevented from worsening despite repetition 
40 of the decoding and encoding processes. 

[0498] The transcoder according to this embodi- 
ment also determines whether or not to reuse the past 
coding parameters based on the picture types con- 
tained in the past coding parameters, thereby enabling 
45 an optimal encoding process. 

[0499] Computer programs for the above processes 

can be- provided by recording them on a recording 

— medium such as a magnetic disc, an optical disc, a 
photo-electro-magnetic disc, or a semiconductor mem- 
so ory, or transmitting them via a network such as the Inter- 
net, an ATM, or a digital satellite so that they can be 
recorded on a user's recording medium. 
[0500] Additionally, according to the transcoder of 
this embodiment, the information for the data set repre- 
ss senting a combination of coding parameters for the past 
encoding processes is described in the encoded stream 
re-coded by the transcoder. This configuration can gen- 
erate stream containing history information that can be 
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transmitted via a transmission path of a small capacity. 
[0501] According to the transcoder of this embodi- 
ment, the plurality of coding parameters used for the 
past encoding processes are selectively combined to 
generate encoding history information, which is then s 
superposed on the encoded stream. As a result, a 
stream capable of restraining image degradation stem- 
ming from re-coding can be transmitted via a small- 
capacity medium. 

[0502] According to the transcoder of this embodl- io 
ment, coding parameters are extracted based on the 
information indicating a data set to generate the re- 
coded encoded stream based on the extracted coding 
parameters. Consequently, a stream capable of 
restraining image degradation stemming from re-coding is 
and of being transmitted to a transmission medium with 
small transmission capacity can be encoded. 
[0503] The transcoder according to this embodi- 
ment records the detected encoding history of the past 
encoding processes on the recording medium, thereby 20 
restraining image quality degradation even if the 
encoded stream is recorded on the recording medium. 
[0504] The transcoder according to this embodi- 
ment detects the encoding history contained in the 
encoded stream reproduced from the recording medium 25 
to multiplex the history with the re-coded encoded 
stream for output, thereby restraining image quality deg- 
radation even if the encoded stream reproduced from 
the recording medium is transcoded again. 

30 

Industrial Applicability 

[0505] The present invention relates to a coding 
system and method, an encoding device and method, a 
decoding device and method, a recording device and 35 
method, and reproducing device and method, and is 
particularly applicable to a transcoder for re-coding an 
encoded stream that have been encoded pursuant to 
the MPEG standard in order to generate an re-coded 
stream having a different GOP (Group of Pictures) or bit 40 
rate. 

Explanation of Reference Numerals 

[0506] 1, 106 ... Encoding device, 2, 102 ... Decod- 43 
ing device, 3 ... Recording medium, 14, 33 ... Frame 
— memory, 17, 32 ... Format converting circuit, 18 ... 
Encoder, 1 9 ... Recording circuit, 30 ... Reproduction cir- 
cuit, 31 ... Decoder, 50 ... Motion vector-detecting cir- 
cuit, 57 ... Quantization circuit, 58 ... Variable-length so 
encoding circuit, 59 ... Transmit buffer, 64, 87 ... Motion 
compensating circuit, 70 ... Controller, 81 ... Reception 
buffer, 82 ... Variable-length decoding circuit, 83 ... 
Quantization circuit, 84 ... IDCT circuit, 100 ... Compu- 
ter, 101 ... Transcoder, 103 ... History information-multi- ss 
plexing device, 104 ... History decoding device, 105 ... 
History information-separating device, 107 ... History 
encoding device, 201 ... User data decoder, 202, 212 ... 



Converter, 203 ... History VLC. 
Claims 

1. A coding system for re-coding a source encoded 
stream, characterized by comprising: 

decoding means for decoding said source 
encoded stream to generate video data and 
extracting from said source encoded stream 
past coding parameters generated by past 
encoding processes; 

encoding means for re-coding said video data 
to generate a re-coded video stream; and 
control means for receiving said past coding 
parameters to control the re-coding process 
carried out by said encoding means, based on 
said past coding parameters, and selectively 
describing said past coding parameters in said 
re-coded stream. 

2. The coding system according to claim 1 , character- 
ized in that in said past coding parameters, said 
control means select coding parameters in macrob- 
locks to describe them in said re-coded stream. 

3. The coding system according to claim 1 , character- 
ized in that in said past coding parameters, said 
control means describes all coding parameters 
relating to a picture unit, in said re-coded stream 
regardless of said application, and selectively 
describes coding parameters in macroblocks, in 
said re-coded stream based on said application. 

4. The coding system according to claim 1 , character- 
ized in that said control means describes said 
selected past coding parameters In said re-coded 
stream as history_stream0 and describes informa- 
tion indicating a data set for said selected past cod- 
ing parameters, in said re-coded stream as 
re_codi ng_stream Jnf6(). 

5. The coding system according to claim 2, character- 
ized in that said re_coding_stream_info() contains 
red_bw_flag and red_bw_indicator as information 
for identifying said data set. 

6. The coding system according to claim 2, character- 
ized in that said data set comprises: 

a plurality of data sets including at least a data 
set for transmitting coding parameters required 
to generate a completely transparent re-coded 
stream; and 

a data set for transmitting coding parameters 
required to generate a relatively transparent re- 
coded stream. 
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7. The coding system according to claim 2, character- 
ized in that said data set comprises: a plurality of 
data sets inciuding at least a data set for transmit- 
ting ail said past coding parameters to said re- 
coded parameters as history_stream(); and 

a data set that does not transmit those of said 
past coding parameters which follow 
picture_data(), to said re-coded parameters as 
h!story_streamO- 

8. The coding system according to claim 2, character- 
ized in that wherein said data set comprises: 

a plurality of data sets including at least a data 
set for transmitting coding parameters in pic- 
tures and in macroblocks to said re-coded 
parameters as history_streamQ; and 
a data set that transmits the coding parameters 
in pictures to said re-coded parameters as 
hlstory_stream(), while not transmitting the 
coding parameters in macroblocks to said re- 
coded parameters as history_stream(). 

9. The coding system according to claim 1 , character- 
ized in that 

said encoding means encodes, during said cur- 
rent encoding process, a reference picture con- 
tained in said input video data, using the 
current picture type assigned to said reference 
picture, and: 

said control means determines whether or not 
said reference picture has been encoded into 
the same picture type as that assigned during a 
past encoding process and controls said cur- 
rent encoding process based on a result of the 
determination. 

10. The coding system according to claim 9, character- 
ized in that based on said determination, said con- 
trol means selects optimal ones of said past coding 
parameters to control the current encoding process 
carried out by said encoding means, based on said 
selected optimal coding parameters. 

-11. The coding system according to claim 9, character- 
ized in that said control means encodes said refer- 
ence picture using one of the past coding 
parameters generated during said past encoding 
process. 

12. The coding system according to claim 9, character- 
ized in that said past coding parameters include 
motion vector information generated during said 
past encoding process, and: 

said encoding means includes motion vector- 



detecting means for detecting motion vector 
information for said reference picture during 
said current encoding process. 

5 13. The coding system according to claim 12, charac- 
terized in that said control means controls operation 
of said motion vector-detecting means based on 
said result of the determination. 

io 14. The coding system according to claim 13, charac- 
terized in that said control means reuses said 
motion vector information included in said past cod- 
ing parameters, Instead of allowing said motion 
vector-detecting means to calculate new motion 

is vector information. 

15. The coding system according to claim 9, character- 
ized in that said control means selects optimal ones 
of said past encoding parameters which corre- 
20 spond to said current encoding process and con- 
trols said current encoding process carried out by 
said encoding means, based on said optical coding 
parameters. 

25 16. The coding system according to claim 9, character- 
ized in that 

said encoding means encodes, during said cur- 
rent encoding process, a reference picture con- 
30 tained in said input video data, using the 

current picture type assigned to said reference 
picture, and 

said control means determines whether or not 
said reference picture has been encoded into 
35 the same picture type as that assigned during a 

past encoding process and select said optimal 
encoding parameters based on a result of the 
determination. 

40 17. The coding system according to claim 15, charac- 
terized in that 

said past coding parameters include prediction 
mode information representing a frame predic- 
45 tion mode or a field prediction mode, and 

said control means controls said current 
encoding process depending on said prediction 
mode information. 

so 18. The coding system according to claim 15, charac- 
terized In that If said reference picture has been 
encoded into the same picture type as said past 
picture type, said control means reuses said predic- 
tion mode information included in said past coding 

55 parameters instead of calculating new prediction 
mode information. 

19. The coding system according to claim 15, charac- 
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terized in that 

said coding parameters include prediction type 
information indicating an intraprediction, a for- 
ward prediction, a backward prediction, or a 5 
bidirectional prediction, and wherein: 
said control means controls said current 
encoding process based on said prediction 
type information. 

u 

20. The coding system according to claim 15, charac- 
terized in that 

said coding parameters include DCT mode 

information representing a frame DCT mode or 15 

a field DCT mode, and wherein: 

said control means controls said current 

encoding process based on said DCT mode 

information. 

20 

21. The coding system according to claim 9, character- 
ized in that said encoding means has generation 
means for generating an MPEG bit stream conform- 
able to the MPEG standard and having a sequence 
layer, a GOP layer, a picture layer, a slice layer, and 25 
a macroblock layer. 

22. The coding system according to claim 21 , charac- 
terized in that 

30 

said control means generates the current cod- 
ing parameters corresponding to said current 
encoding process carried out by said encoding 
means, and 

said control means describes said current cod- 35 
ing parameters in said picture, slice, and mac- 
roblock layers of said re-coded stream while 
selectively describing said past coding param- 
eters in a user data area of the picture layer of 
said re-coded stream. 40 

23. The coding system according to claim 9, character- 
ized in that said control means describes said past 
coding parameters in said re-coded stream as 
history_stream(). 45 

24. ~A~coding system for executing a re-coding process 

on a source encoded stream, characterized by 
comprising: 

50 

decoding means for decoding said source 
encoded stream to generate decoded video 
data and extracting from said. source encoded 
stream past coding parameters contained in 
said source encoded stream; 55 
encoding means for executing the re-coding 
process on said decoded video data to gener- 
ate re -coded stream; and 



control means for selecting ones of said past 
coding parameters which are required for an 
application connectiveiy following said encod- 
ing means and describing the selected past 
coding parameters in said re-coded stream. 

25. A coding method for executing a re-coding process 
on a source encoded stream, characterized by 
comprising the steps of: 

decoding said source encoded stream to gen- 
erate decoded video data and extracting from 
said source encoded stream past coding 
parameters contained in said source encoded 
stream; 

executing the re-coding process on said 
decoded video data to generate re-coded 
stream; and 

selecting ones of said past coding parameters 
which are required for an application connec- 
tiveiy following said encoding means and 
describing the selected past coding parame- 
ters in said re-coded stream. 

26. A coding system for executing a re-coding process 
on a source encoded stream, characterized by 
comprising: 

decoding means for decoding said source 
encoded stream to generate decoded video 
data and extracting from said source encoded 
stream past coding parameters contained in 
said source encoded stream; 
encoding means for executing the re-coding 
process on said decoded video data to gener- 
ate a re-coded stream; and 
control means for selectively describing said 
past coding parameters in said re-coded 
stream while describing in said re-coded 
stream a flag or/and an indicator indicating a 
data set for the past coding parameters 
described in said re-coded stream. 

27. A coding method for executing a re-coding process 
on a source encoded stream, characterized by 
comprising: 

a decoding step of decoding said source 
encoded stream to generate decoded video 
data and extracting from said source encoded 
stream past coding parameters contained in 
said source encoded stream; 
an encoding step of executing the re-coding 
process on said decoded video data to gener- 
ate a re-coded stream; and 
a control step of selectively describing said 
past coding parameters in said re-coded 
stream while describing in said re-coded 
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stream a flag or/and an indicator indicating a 
data set for the past cooing parameters 
described in said re-coded stream. 

28. A coding system for executing a re-coding process 
on a source encoded stream, characterized by 
comprising: 

decoding means for decoding said source 
encoded stream to generate decoded video 
data and placing in said source encoded 
stream past coding parameters transmitted as 
history_streamO; 

encoding means for executing the re-coding 
process on said decoded video data to gener- 
ate a re-coded stream; and 
control means for describing information on 
said past coding parameters in said re-coded 
stream as history_stream() while describing 
information on said re-coded stream in said re- 
coded stream as re_codlng_streamJnfo(). 

29. A coding method for executing a re-coding process 
on a source encoded stream, characterized by 
comprising: 

a decoding step of decoding said source 
encoded stream to generate decoded video 
data and placing in said source encoded 
stream past coding parameters transmitted as 
history_stream(); 

an encoding step of executing the re-coding 
process on said decoded video data to gener- 
ate a re-coded stream; and 
a control step of describing information on said 
past coding parameters in said encoded 
stream as history_stream() while describing 
information on said re-coded stream in said re- 
coded stream as re_coding_stream_info(). 

30. A coding system for executing a re-coding process 
on a source encoded stream, characterized by 
comprising: 



decoding means for decoding said source 
encoded stream to generate decoded video 
-data- and- placing in-said source encoded- 
stream past coding parameters transmitted as 
history_stream(); 

encoding means for executing the re-coding 
process on said decoded video data to gener- 
ate a re-coded stream; and 
control means for selectively describing said 
past coding parameters in said re-coded 
stream as history_stream() while describing in 
said re-coded stream as 

re_coding_stream_info(), information on a data 
set for the past coding parameters described in 



said re-coded stream. 

31. A coding method for executing a re-coding process 
on a source encoded stream, characterized by 

5 comprising: 

a decoding step of decoding said source 
encoded stream to generate decoded video 
data and placing in said source encoded 
io stream past coding parameters transmitted as 

histo ry_stream () ; 

an encoding step of executing the re-coding 
process on said decoded video data to gener- 
ate a re-coded stream; and 

is a control step of selectively describing said 

past coding parameters In said re-coded 
stream as history_stream() while describing in 
said re-coded stream as 

re_codingL_streamJnfoO, information on a data 

20 set for the past coding parameters described in 

said re-coded stream. 

32. An encoding device for encoding video data, char- 
acterized by comprising: 

25 

means for receiving past coding parameters for 
past encoding processes carried out on said 
video data; 

encoding means for encoding said video data 
30 to generate a re-coded stream; and 

control means for receiving the past coding 
parameters for the past encoding processes 
carried out on said video data and selectively 
describing said past coding parameters in said 
35 re-coded stream while describing in said re- 

coded stream information indicating a data set 
for said past coding parameters described in 
said re-coded stream. 

40 33. An encoding method for encoding video data, char- 
acterized by comprising: 

a step of receiving past coding parameters for 
past encoding processes carried out on said 

45 video data; 

an encoding step of encoding said video data 

- to generate a re-coded stream;-and — ■ — 

a control step of receiving the past coding 
parameters for the past encoding processes 

so carried out on said video data and selectively 

describing said past coding parameters In said 
re-coded stream while describing in said re- 
coded stream information indicating a data set 
for said past coding parameters described in 

55 said re-coded stream. 

34. An encoding device for encoding video data, char- 
acterized by comprising: 
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means for receiving past coding parameters for 
past encoding processes earned out on said 
video data; 

encoding means for encoding said video data 
to generate a recoded stream; and 
control means for describing said past coding 
parameters in said re-coded stream as 
hlstory_stream() while describing in said re- 
coded stream as re_coding_stream_info(), 
information on said past coding parameters 
described in said re-coded stream. 

35. An encoding method for encoding video data, char- 
acterized by comprising: 

a step of receiving past coding parameters for 
past encoding processes carried out on said 
video data; 

an encoding step of encoding said video data 
to generate a re-coded stream; and 
a control step of describing said past coding 
parameters in said re-coded stream as 
history_stream() while describing in said re- 
coded stream as re_coding_stream_Info(), 
information on said past coding parameters 
described in said re-coded stream. 

36. An encoding device for encoding video data, char- 
acterized by comprising: 

means for receiving past coding parameters for 
past encoding processes carried out on said 
video data; 

re-coding means for executing the re-coding 
process on said video data to generate a re- 
coded stream; and 

control means for controlling said re-coding 
process so that said coding parameters are 
described in said re-coded stream as 
history_stream() while information on a data 
set for the coding parameters described in said 
re-coded stream is described in said re-coded 
stream as re_coding_strearn_info0. 

37. An encoding method for encoding video data, char- 
acterized by comprising: 



20 



25 



a step of receiving past coding parameters for 
past encoding processes carried out on said 
video data; 

an encoding step of executing the re-coding 
process on said video data to generate a re- 
coded stream; and 

a control step of controlling said re-coding 
process so that said past coding parameters 
are described in said re-coded stream as 
history_stream() while information on a data 
set for the coding parameters described in said 



re-coded stream is described in said re-coded 
stream as re_coding_stream_jnfo(). 

38. An encoding device for encoding video data, char- 
5 acterized by comprising: 

means for receiving past coding parameters for 
past encoding processes carried out on said 
video data; 

10 encoding means for encoding said video data 

to generate a re-coded stream; and 
control means for receiving the past coding 
parameters for the past encoding processes 
carried out on said video data and selective ly 

is superposing said past coding parameters in 

said re-coded stream white superposing In said 
re-coded stream information indicating a data 
set for said past coding parameters described 
in said re-coded stream. 

39. An encoding method for encoding video data, char- 
acterized by comprising: 

a step of receiving past coding parameters for 
past encoding processes carried out on said 
video data; 

an encoding step of encoding said video data 
to generate a re-coded stream; and 
a control step of receiving the past coding 
parameters for the past encoding processes 
carried out on said video data and selectively 
superposing said past coding parameters in 
said re-coded stream while superposing in said 
re-coded stream information indicating a data 
set for said past coding parameters described 
in said re-coded stream. 

40. An encoding device for encoding video data, char- 
acterized by comprising: 

means for receiving past coding parameters for 
past encoding processes carried out on said 
video data; 

encoding means for encoding said video data 
to generate a re-coded stream; and 
control means for superposing said past coding 

parameters in said redded -streanv as 

hlstory_stream() while superposing in said re- 
coded stream as re_coding_stream_infoO, 
so information on said past coding parameters 

described in said re-coded stream. 

41. An encoding method for encoding video data, char- 
acterized by comprising: 

55 

a step of receiving past coding parameters for 
past encoding processes carried out on said 
video data; 
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an encoding step of encoding said video data 
to generate a re-coded stream; and 
a control step of superposing said past coding 
parameters in said re-coded stream as 
history_streamO while superposing in said re- s 
coded stream as re_codingL_streamJnfo(), 
information on said past coding parameters 
described In said re-coded stream. 

42. A decoding device for decoding an encoded w 
stream, characterized by comprising: \ 

) . 

decoding means for decoding said encoded 
stream to generate decoded video data; 
means for extracting from said encoded stream is 
information on a data set for the past coding 
parameters superposed in said encoded 
stream; and 

means for extracting said past coding parame- 
ters from said encoded stream based on said 20 
information on the data set. 

43. A decoding method for decoding an encoded 
stream, characterized by comprising: 

25 

a decoding step of decoding said encoded 
stream to generate decoded video data; 
a step of extracting from said encoded stream 
information on a data set for the past coding 
parameters superposed in said encoded 30 
stream; and 

a step of extracting said past coding parame- 
ters from said encoded stream based on said 
information on the data set 

35 

44. A decoding device for decoding an encoded 
stream, characterized by comprising: 

decoding means for decoding said encoded 
stream to generate decoded video data; 40 
means for extracting from said encoded stream 
information on a data set indicating the past 
coding parameters superposed in said 
encoded stream; 

means for extracting said past coding parame- 45 
ters from said encoded stream based on said 

information on the data set; and 

transmission means for transmitting said, 
decoded video data and said extracted past 
coding parameters. 50 

45. A decoding method for decoding an encoded 
stream* characterized by comprising: 

a decoding step of decoding said encoded ss 
stream to generate decoded video data; 
a step of extracting from said encoded stream 
information on a data set indicating the past 



coding parameters superposed in said 
encoded stream; 

a step of extracting said past coding parame- 
ters from said encoded stream based on said 
information on the data set; and 
a transmission step of transmitting said 
decoded video data and said extracted past 
coding parameters. 

46. A decoding device for decoding an encoded 
stream, characterized by comprising: 

'decoding means for decoding said encoded 
stream to generate decoded video data; 
means for extracting from said encoded stream 
a flag or/and an Indicator described In said 
encoded stream as re_coding__stream_infor(); 
means for extracting said past coding parame- 
ters from said encoded stream based on said 
flag or/and indicator; and 
transmission means for transmitting said 
decoded video data and said extracted past 
coding parameters. 

47. A decoding method for decoding an encoded 
stream, characterized by comprising: 

a decoding step of decoding said encoded 
stream to generate decoded video data; 
a step of extracting from said encoded stream 
a flag or/and an indicator described in said 
encoded stream as re_coding_stream_inforO; 
a step of extracting said past coding parame- 
ters from said encoded stream based on said 
flag or/and indicator; and 
a transmission step of transmitting said 
decoded video data and said extracted past 
coding parameters. 

48. A recording device for recording an encoded 
stream on a recording medium, characterized by 
comprising: 

decoding means for decoding a source 
encoded stream to generate video data and 
extracting from said source encoded stream 

coding—parameters— generated during— past 

encoding processes; 

encoding means for executing a re-coding 
process on said video data to generate a re- 
coded video stream; 

control means for controlling said re-coding 
process based on said past coding parameters 
and selectively describing said past coding 
parameters in said re-coded stream; and 
means for recording said re-coded stream on 
the recording medium. 
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49. A recording method for recording an encoded 
stream on a recording medium, characterized by 
comprising: 

a decoding step of decoding a source encoded 
stream to generate video data and extracting 
from said source encoded stream coding 
parameters generated during past encoding 
processes; 

an encoding step of executing a re-coding 
process on said video data to generate a re- 
coded stream; 

a control step of controlling said re-coding 
process based on said past coding parameters 
and selectively describing said past coding 
parameters in said re-coded stream; and 
a step of recording said re-coded stream on the 
recording medium. 

50. A recording device for recording an encoded 
stream on a recording medium, characterized by 
comprising: 

means for receiving past coding parameters for 
past encoding processes carried out on sup- 
plied video data; 

encoding means for encoding said video data 
to generate a re-coded stream; 
control means for describing said past coding 
parameters in said re-coded stream as 
history_stream() while describing in said re- 
coded stream as re_codIng_stream_jnfo(), 
information on said past coding parameters 
described in said re-coded stream; and 
means for recording said re-coded stream on 
the recording medium. 

51. A recording method for recording an encoded 
stream on a recording medium, characterized by 
comprising: 
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20 



25 



30 



40 



a step of receiving past coding parameters for 
past encoding processes carried out on sup- 
plied video data; 

an encoding step of encoding said video data 
to generate a re-coded stream; 
- a control step of describing said past coding - 
parameters in said re-coded stream as 
history_stream() while describing in said re- 
coded stream as re_coding_stream_info(), 
Information on said past coding parameters 
described in said re-coded stream; and 
a step of recording said re-coded stream on the 
recording medium. 



52. A reproducting device for reproducing an encoded 
stream from a recording medium, characterized by 
comprising: 



45 



reproduction means for reproducing the source 
encoded stream from said recording medium; 
decoding means for decoding said source 
encoded stream to generate video data and 
extracting from said source encoded stream 
coding parameters generated during past 
encoding process; 

encoding means for executing the re-coding 
process on said video data to generate a re- 
coded video stream; and 
control means for controlling said re-coding 
process based on said past coding parameters 
and selectively describing said past coding 
parameters in said re-coded stream. 

53. A reproduction method for reproducing an encoded 
stream from a recording medium, characterized by 
comprising: 

a reproduction step of reproducing the source 
encoded stream from said recording medium; 
a decoding step of decoding said source 
encoded stream to generate video data and 
extracting from said source encoded stream 
coding parameters generated during past 
encoding process; 

an encoding step of executing the re-coding 
process on said video data to generate a re- 
coded video stream; and 
a control step of controlling said re-coding 
process based on said past coding parameters 
and selectively describing said past coding 
parameters in said re-coded stream. 

54. A reproducting device for reproducing an encoded 
stream from a recording medium, characterized by 
comprising: 



reproduction means for reproducing the 
encoded stream from said recording medium; 
decoding means for decoding said encoded 
stream to generate decoded video data; 
means for extracting from said encoded stream 
a flag or/and an indicator described in said 
encoded stream as re_coding_stream_jnfo(); 
means for extracting said past coding parame- 
ters from said encoded stream based on said- 
flag or/and indicator; and 
transmission means for transmitting said 
decoded video data and said extracted past 
coding parameters. 



55. A reproduction method for reproducing an encoded 
stream from a recording medium, characterized by 
comprising: 

a reproduction step of reproducing the 
encoded stream from said recording medium; 



49 



97 EP 1 069 779 A1 

a decoding step of decoding said encoded 
stream to generate decoded video data; 
a step of extracting from said encoded stream 
a flag or/and an indicator described in said 
encoded stream as re_co dlng_streamj nf o 0 ; 5 
a step of extracting said past coding parame- 
ters from said encoded stream based on said 
flag or/and indicator; and 
a transmission step of transmitting said 
decoded video data and said extracted past w 
coding parameters. 
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history stream(40-5) 



* 


OIXS 


T : 1 

value 










32 


00000 IBS 




4 


7 


picTure M aispiay_exieri3ion _present_nag 


1 




rrarn e_cenire_nortxoniai — OTTset_ l 


16 


• 


fiiarKer^ on 


1 


1 


1 rrame^cen tre_veruc2u_ofTset_ 1 


16 




mairter^bft 


1 


1 


f r ame_centre - ,horizontal_offset_2 


16 




marker_bit 


1 


1 


frame_centre_vertical_offset_2 


16 




marker_bit 


1 


^ 1 


frame_centre_horizontaLotfset_3 


16 




marker_bit 


1 


1 


frafne_centre_verticaJ_offset_3 


16 




marker_bit 


6 


3F 








re_coding_strearn_inforrnation 






user_data_start_code 


32 


00O0O1B2 


re jcod In g^jstr eam — Fnf o J O 


- 16 


91 EC 


red_bwjfag 


1 




redj bw_ind icator 


2 




rnarxcr^Dii 


5 


1F 














user rlntsfc ^frstrf rrvio 


32 


000001 B2 


f user Hats 


2048 










while(rnacrobfock i=macroblock_count){ 






macroblock 






macro block__address_h 


8 




macroblock_address_v 


8 




slice Jieader j*esent_flag 


1 




skipped _macroblock_flag 


1 




marker_bit 


1 


1 








macroblock modes( ) 






j macroblock_quant . j 


1 




r macroblock_motJonJorward 


1 




macroblock_motfon_backward 


1 




macroblockjiattefn 


1 




macroblockjntra 


1 





FIG. 44 



94 



EP 1 069 779 A1 



history stream(40-6) 
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copyright_extension( ) { 


No of bits 


in I idllUI IKm 


exteraon_start_codejdentrfier 


A 


uirnsur 


copyrigftt_fla« 


1 


bslbf 


copyright Jdentffier 


8 


uimsbf 


origlnal_or_copy 


1 


bslbf 


| reserved 


7 


uimsbf 


marker Jbit 


1 


bslbf 


copyright_ru*nber_ 1 


20 


uimsbf 


marker J>it 


1 


bslbf ] 


copyrfght_ru*Tt>er_2 


22 


uimsbf 


marker_bft 


1 


bslbf 


copyright_number_3 


22 


uimsbf 


next_start_code( ) 






} 







FIG. 57 
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picture_data( ) ( 
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macroblock( ) { 
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macrob!ock_modes( ) { 
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macroblock_motkxi_backward 












Description 


1 


0 


1 


0 


0 


Intra 


01 


1 


1 


0 


0 


Intra, Quant 



FIG. 65 



macroblock.type VLC code 




macroblock_quant 






dct_type_fla£ 








macroblock_motion_forward 










rnacroblock_rnotion_backward 












Description 


1 


0 


1 


1 


0 


MC. Coded 


01 


0 


1 


0 


0 


No MC, Coded 


001 


0 


0 


0 


0 


MC, Not Coded 


0001 1 


0 


1 


0 


0 


Intra 


0001 0 - 


1 


1 


1 


o 


... -MC, Coded. Quant 


0000 1 


1 


1 


0 


0 


No MC, Coded. Quant 


0000 01 


1 


1 


0 


0 


Intra, Quant 



FIG. 66 



108 



EP 1 069 779 A1 



macrobtock^t ype V L C code 

macroblock^quant 

d ctjtypejflag 

maca^lock_motion_forward 
macroblock_motion_backward 
[~ Description 



10 


0 


0 


0 


0 


Interp, Not Coded 


11 


0 


1 


1 


1 


Interp, Coded 


010 


0 


0 


0 


0 


Bwd, Not Coded 


011 


0 


1 


0 


1 


Bwd, Coded 


0010 


0 


0 


0 


0 


Fwd, Not Coded 


0011 


0 


1 


1 


0 


Fwd. Coded 


0001 1 


0 


1 


0 


0 


Intra 


0001 0 


1 


1 


1 


1 


Interp, Coded, Quant 


0000 1 1 


1 


1 


1 


0 


Fwd, Coded, Quant 


0000 10 


1 


1 


0 


1 


Bwd, Coded, Quant 


0000 01 


1 


1 


0 


0 


Intra, Quant 



FIG. 67 



109 



EP 1 069 779 A1 
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